@JonathanGeuter
Pinned post
Some things that were useful when my agent and I were building the data for this model:View quoted post on X
- Check how well you can guess the answer from the options alone. One source looked like a +4.5 gain, but it was mostly "the longest option is right". Real gain was more like +2.3
- Weight wrong options by how common they are. arXiv labels with uniform wrong options got +18 over chance without reading any text
- Keep one writing style across classes. With one class written by humans and the other by an LLM, the model just learned to spot LLM style
- Shuffle the options per example. Models that read all the options together can flip answers when you reorder them (12-23% in Crivo)
- Try different mixes, but with more than one seed. Single runs differed by up to ~8 points on one test set. Adding diverse sources gave the same lift as 4x the rows and steps, at about a quarter of the compute
- Check what your trainer silently drops. Ours skipped 53k one-label rows, the only verification data in the mix
Data, with source, labeller and licence on every row: huggingface.co/datasets/davanβ¦
Here's what happens to speculative decoding when the prompt gets long.
@GoogleAI @googlegemma
Gemma 4 26B on one RTX 3090, 28k-token prompt:
DFlash: 0.66x. Slower than not speculating at all.
Uno: 1.30x. Still pulling ahead.
That's 2x DFlash. Lossless. Same card, same model, same AWQ 4-bit weights, faster than plain at every length from 2k to 28k.
The drafter behind it? A tiny 10k open prompts, trained on a single GPU. No cluster, no mountain of data.
Uno runs in @vllm_project today: github.com/brntech/vllm-uβ¦
The long-context build and the new drafter land in the next release, with the @IFM_AI team.View quoted post on X
π Uno is on Hugging Face, go try it out: huggingface.co/IFM/K2-Horizonβ¦View quoted post on X
