Twitxy Saved

murat 馃崶

Verified account This account is verified. Learn more

@mayfer

Vancouver, BC muratayfer.com Joined March 2008
9.7K Following 20K Followers 16K Posts

Latest posts 5

murat 馃崶 Verified account @mayfer
now that we're interpreting monosemantic neurons it's time to bring this up again. fast memory recall & mechinterp are the same problem. let's figure out activation-triggered indexing. i think it's doable
murat 馃崶 Verified account @mayfer Dec 21, 2025
imo e2e mechanistic interpretability has the highest chance of leading to e2e memory recall by providing activation-triggered indexing and search
View quoted post on X
murat 馃崶 Verified account @mayfer
my "abstraction neurons as embeddings" method beats all SOTA embedding search methods on "word similarity" but loses very badly on sentence similarity and retrieval

i'd say this is expected given it uses 16k neurons out of 4B but i'm really pleased how well it works for words
murat 馃崶 Verified account @mayfer Oct 22
i built an embedding engine by composing vectors from common abstraction neuron activations (h/t @nickcammarata "already clean" essay)

and it behaves more consistently especially when doing embedding math. less distracted by spelling/formatting and more focus on abstractions present

it's not always better, but generally at least on par. i have not seen it breakdown into returning junk like regular embedding models do yet. i am gonna try running some vector search evals on it to see how it performs
View quoted post on X