Twitxy Saved

Jack Cook

Verified account This account is verified. Learn more

@jackcookjack

mts @thinkymachines | ex-phd student @mit | systems for ml
San Francisco, CA jackcook.com Joined January 2011
555 Following 2.2K Followers 169 Posts

Latest posts 5

Pinned post

Jack Cook Verified account @jackcookjack
Inkling 2 Small is our first model trained partly in NVFP4, with a mixed-precision recipe that enables a great tradeoff between speed and accuracy. Check out our new post to see how we did it, and the lessons we learned along the way!
Thinking Machines Verified account @thinkymachines Nov 13
Training with fewer bits is faster, but quantization error can interfere with learning.

Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout.
thinkingmachines.ai/blog/end-to-en…
View quoted post on X
23 replies 32 reposts 262 likes 24K views View on X
Jack Cook Verified account @jackcookjack
Loved this post from @humansand on NVFP4 for RL, and glad to see Four Over Six (4/6) helped make the recipe more stable! Really nice visualizations + profiling analysis as well
humans& @humansand Jul 10
At humans&, we train models from the long-term impacts of their interactions with people. This requires prioritizing long-horizon multi-agent RL. We've developed and are excited to share an open-source, hardware-native 4-bit RL recipe, significantly accelerating training
View quoted post on X
0 replies 4 reposts 61 likes 5.4K views View on X
Jack Cook Verified account @jackcookjack
Now seems like a good time to share that I’ve recently joined @thinkymachines to work on pretraining! Very excited to work on the future of human-AI collaboration with this amazing team.
Thinking Machines Verified account @thinkymachines May 11
People talk, listen, watch, think, and collaborate at the same time, in real time. We've designed an AI that works with people the same way.

We share our approach, early results, and a quick look at our model in action.

thinkingmachines.ai/blog/interacti…
View quoted post on X
12 replies 5 reposts 139 likes 12K views View on X
Jack Cook Verified account @jackcookjack
NVFP4 allows models to be quantized to 4 bits without too much performance degradation, but can we push 4-bit performance even further?

Today, we're releasing a new class of low-precision block-scaled data types that natively adapt to your input data: for 4-bit quantization, IF4 (Int/Float 4) allows each scaled group of 16 values to be saved as FP4 or INT4 depending on which option offers less error. Selections are recorded using the scale factor’s sign bit, which is unused in NVFP4, allowing IF4 to offer better performance with no memory overhead!

Our data types provide better downstream accuracy in LLMs, they can be implemented efficiently in next-generation hardware accelerators, and they reveal some interesting insights about low-bit quantization! 🧵
15 replies 81 reposts 440 likes 54K views View on X