Pinned post
Optimized-parquet is now on @huggingface, this is a new era for datasets !
here is why it's amazing and how it works:
here is why it's amazing and how it works:
@lhoestq
Pinned post
A pretty important new primitive just landed in ๐ค Transformers: **decision models**.View quoted post on X
I think this could matter much more than adding support for any single model.
Today, different decision models all approach "making a decision" differently:
- @typesafeaiโs Jev, the OG: typed decisions + calibrated probabilities, single forward
- @AutoTrustAIโs JEV builds open System One models around a learned decision block, w/ multimodal.
- @jaredpalmerโs KEV uses a pointer head over the options, sharing the computation of the state across questions.
- @Cloudflareโs CLEF uses a joint schema head that routes evidence between the state, questions and options, scoring the whole schema together.
- @perplexity_aiโs Decider uses a separate 255-way readout on top of Qwen, with non-causal attention in its full-attention layers.
- @liquidaiโs d1 is exploring small, multimodal decision models that can run all the way down to the edge.
- And @XunzhuoLiu's Semantic Router team is pushing the serving side hard with System One + the D3 family: one typed interface for choice, score and yes/no decisions, with models from sub-1B to 27B.
That experimentation is good, and needed! But without a common interface, every training framework, inference engine and evaluation tool has to understand every implementation separately.
So Transformers is introducing a shared decision-model layer:
`DecisionConfig`
`pipeline("decision")`
and, most importantly:
`model.decide(...)`
The idea is **not** to make every decision model work the same internally, but for them to speak the same language to the ecosystem around them.
This means that, without running custom or remote code:
- inference engines can serve them through one interface
- training tools can build against common primitives.
- the Hub can package them consistently.
- evaluations can compare them without model-specific glue everywhere.
This is the part of transformers I'm the most excited about: to grow alongside the ecosystem and the many ideas; and to make all of them more usable by everyone.
github.com/huggingface/trโฆ