Twitxy Saved

Evan Hernandez

Verified account OpenEvidence

@evanqed

ph.d. @mit | building @openevidence (we're hiring!) | formerly @google @uwmadison | #nlproc and loud music
Miami, FL evandez.com Joined March 2020
155 Following 462 Followers 34 Posts

Latest posts 5

Evan Hernandez Verified account @evanqed
I will be at ICLR this week to share this work! Stop by and say hi, or let me know if you want to chat about LM interpretability :)
Jacob Andreas @jacobandreas May 6, 2024
Linearity of relation decoding in transformer LMs (led by *newly-minted Dr.* @evanqed and @arnab_api)

> Understanding how LMs transform internal knowledge representations to output predictions

arxiv.org/abs/2308.09124
View quoted post on X
1 replies 2 reposts 12 likes 3.3K views View on X
Evan Hernandez Verified account @evanqed
It's been awesome to see this work come together - an agentic interpretability system that iteratively experiments on a model to understand how it works!

I believe "LMs with tools" is a powerful paradigm for auto interp. Lots of room for new tools + LMs to increase the scope!
Tamar Rott Shaham @TamarRottShaham May 3, 2024
Accepted to #ICML2024!! 馃殌

Meet MAIA- a Multimodal Automated Interpretabilty Agent that helps users understand AI systems.

Given a user query (eg "label a model鈥檚 feature"), MAIA designs experiments iteratively by forming and updating hypotheses based on experimental results.
View quoted post on X
0 replies 5 reposts 13 likes 2.5K views View on X
Evan Hernandez Verified account @evanqed
Proud of Eric + team for pushing this work! You don't need a giant LM for everything. Relatively small (< 1B param), specially pretrained language models are effective for processing specialized text like EHR notes. Can beat ICL+big models with only O(100) FT data points
Recent results have suggested that LLMs encode a surprising amount of clinical knowledge. These results raise an important question about the role of smaller specialized clinical models: do we still need clinical LMs? We explore this question in our paper: arxiv.org/abs/2302.08091
View quoted post on X