What if fine-tuning took a paragraph and a few seconds?
We trained a meta-model that writes the weight update for Qwen3.5-2B from a one-paragraph task description, in seconds on one small GPU, no gradient step.
Across 75 public benchmarks, the update written from just one paragraph improved the base model's score on 59 of them and lowered it on 8. For example, ComVE jumped from 50.0 to 91.1 percent, three-object logical deduction went from 46.4 to 82.0, and on GSM8K with the final number only, from 5.7 to 18.6, ahead of a LoRA trained on up to 1,400 rows of GSM8K (12.7).
Post, figures and every benchmark: roicohen.org/blog/compress-…
We trained a meta-model that writes the weight update for Qwen3.5-2B from a one-paragraph task description, in seconds on one small GPU, no gradient step.
Across 75 public benchmarks, the update written from just one paragraph improved the base model's score on 59 of them and lowered it on 8. For example, ComVE jumped from 50.0 to 91.1 percent, three-object logical deduction went from 46.4 to 82.0, and on GSM8K with the final number only, from 5.7 to 18.6, ahead of a LoRA trained on up to 1,400 rows of GSM8K (12.7).
Post, figures and every benchmark: roicohen.org/blog/compress-…
0:30