Pinned post
Finally releasing the results of the arch ablations I was running with bux over the weekend x.com/i/article/2116…
Nikita Pavlichenko
Verified account This account is verified. Learn more@nv_pavlichenko
pre-training @cohere | prev. head of mellum @jetbrains
222 Following 1.4K Followers 296 Posts
Latest posts 5
Happy to share that I’ve joined @cohere pre-training team as a member of technical staff!
I’ve been following them from 2022 and I’m very excited that I can finally take part in training the models
Today was my first day in Berlin office and already very bullish
I’ve been following them from 2022 and I’m very excited that I can finally take part in training the models
Today was my first day in Berlin office and already very bullish
Today was my last day at @jetbrains. It’s very bitter sweet for me because I had a very rare chance to build the llm team (which became Mellum) from ground up. Probably will never have such an experience again. When I joined, except from local completion, there were practically nobody who knew how to train models. We needed to learn everything ourselves from first principles.
The first big pre-training run was launched by essentially three people: @ivanbondyrev, Marko, and me. Now the team is still super small but one of the few who do the whole model cycle from pre-training data and arch to post-training and evals. They were the locomotive of the company transformation from making good code editors to being a big player in the AI era.
Wishing them big luck! Hope they’ll surprise everyone with a new release in a good way.
Now gonna enjoy my two days of being unemployed. Stay tuned!
P.S. a photo of me pushing a start button of Mellum2 pre-training in an Uzbek restaurant. Marko’s babysitting load-balancing loss.
The first big pre-training run was launched by essentially three people: @ivanbondyrev, Marko, and me. Now the team is still super small but one of the few who do the whole model cycle from pre-training data and arch to post-training and evals. They were the locomotive of the company transformation from making good code editors to being a big player in the AI era.
Wishing them big luck! Hope they’ll surprise everyone with a new release in a good way.
Now gonna enjoy my two days of being unemployed. Stay tuned!
P.S. a photo of me pushing a start button of Mellum2 pre-training in an Uzbek restaurant. Marko’s babysitting load-balancing loss.
oh no, all these residual streams things seem to work on my ablations. bad day to be a performance engineer
There is probably nothing wrong with the soverign ai positioning and a lab post-training Kimi. Both have their own place: see Mistral and Cursor Composer. But it's kinda either this or that?