just grateful of being part of the Mila ecosystem!
Diego Calanzone
@diegocalanzone
« artificia docuit fames » // phd at @Mila_Quebec, core RL + DL4Science // ex AI grad @UniTrento
1K Following 299 Followers 779 Posts
Latest posts 5
this is bad narrative
We're live Man vs. Machine x.com/i/broadcasts/1…View quoted post on X
massive work by @antirez. DwarfStar4 brought a 1.6T parameters model to as little as 128GB of vram on a local machine, with quantization etc.
I’m positive this is a step towards personal, capable, AI systems
antirez.com/news/165
I’m positive this is a step towards personal, capable, AI systems
antirez.com/news/165
Fantastic analysis! It is time to shed some more light on alternative tokenization schemes. We already saw with Goomba Lab's H-Nets that not all languages are tokenized equally, calling for a data-driven approach perhaps?
We present Compute Optimal Tokenization! 🔡
Common in LLM scaling works stick to one tokenizer, sweeping data/model size.
But what happens when we control the tokenizer’s compression rate (bytes/token)?
Here we sweep tokenizers, params, and data across compute budgets: [1/N]View quoted post on X
what happened to the “limitations” section in deep learning papers? Talking about LLM-related research especially.
Shouldn’t self-criticism encourage more constructive feedback and better follow-up research?
Shouldn’t self-criticism encourage more constructive feedback and better follow-up research?