rohan anil

Core Automation

@_arohan_

co-founder @coreauto - and continuing to aspire to understand deep learning.

Joined December 2017

2.4K Following 46K Followers 12K Posts

Latest posts 5

Pinned post

rohan anil @_arohan_ Apr 19

It turns out multi step backpropaganda is better.

paper has a beautiful way of improving backpropagation. One iteration cleanly gets us backprop, multiple iterations get us a preconditioned update.

rohan anil Verified account @_arohan_ Apr 19

@LinYorker @ryu0000000001 @weijie444 arxiv.org/abs/2106.06199

Same update here
View quoted post on X

rohan anil @_arohan_ 4h

Weight space programming could be a podcast name, it will go hard!

Few researchers and a whiteboard.

Dimitris Papailiopoulos Verified account @DimitrisPapail 5h

Weight-space programming 2023 edition: two grad students, a whiteboard, and a LoOpEd TrAnFfoRmEr that ran assembly.

Respect to @AngelikiGiannou and @shashank_r12 for riffing this hard on this before it was cool.

(yes, I’ve found another excuse to retweet this for the nth time)
View quoted post on X

rohan anil @_arohan_ 6h

This is also going to get lot of young scientists upset because they stopped developing their insights thinking scaling transformers alone is sufficient and felt defeated due to “the bitter lesson”

Richard Sutton Verified account @RichardSSutton 7h

Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
View quoted post on X

rohan anil @_arohan_ 6h

This is the way

Richard Sutton Verified account @RichardSSutton 7h

Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
View quoted post on X

rohan anil @_arohan_ 20h

I wonder how easy it would be to identify which models generated code that later had CVEs.

Could we use their vulnerability rates to set a minimum security bar for models used in foundational software? That could rule out older or more vulnerability-prone models that fail to meet the bar.

Otherwise, the natural pressure is to optimize for cheaper tokens.