Data selection is just as critical as the training itself. But how do you choose which samples to use?
Our new work explores what works and what doesn't when selecting a subset of instructions to tune on. Awesome thread by @nihalcanrun below! 👇
Our new work explores what works and what doesn't when selecting a subset of instructions to tune on. Awesome thread by @nihalcanrun below! 👇
Targeted instruction tuning for LLMs involves selecting a subset of instructions from a candidate pool using a small query set from target tasks. Despite growing interest, we still lack guidance on what to select. Our new preprint brings clarity to this space (thread 👇).View quoted post on X