Twitxy Saved

Cyrus

Verified account Decagon

@cyrusasg

research lead @decagonai
post-training, inference, mlsys
do good together with the many; share skill so that the many may be skilled.
San Francisco, CA Joined March 2022
329 Following 950 Followers 77 Posts

Latest posts 5

Cyrus Verified account @cyrusasg
super fun being a part of this project!

open models and inference are core to how we do research at decagon
Modal @modal 4d
Is open, accessible intelligence truly achievable?

We teamed up with @Speakeasy_Films to follow the journeys and evolutions of major open-source projects like @sgl_project, @vllm_project, and more to see how they think about open-source in an increasingly closed world.
View quoted post on X
1 replies 2 reposts 28 likes 991 views View on X
Cyrus Verified account @cyrusasg
inkling 2 small is most interesting for its self-proclaimed “post-trainability.” but what does that even mean? i’d frame it as plasticity: how easily the model picks up new behavior without losing what it already knows.

for RL, it’s mostly about the base model’s success rate on your tasks. group-based RL reweights what the model already does, so the right answer has to show up sometimes. if every rollout in a group gets the same reward, there’s no gradient. an exploratory base with good support and reasonable entropy keeps more groups mixed.

another point is how cheaply the model moves. aghajanyan et al. (arxiv.org/abs/2012.13255) showed that better-pretrained models have lower intrinsic dimension for fine-tuning, so the same task needs fewer effective parameters, making LoRA more effective.

the other consideration is how sensitive pretrained weights are to being modified. springer et al. found extended pretraining can make models harder to fine-tune and yield worse final performance (arxiv.org/abs/2503.19206). watts et al. tie this to sharpness, and flatter minima (SAM, higher LR, shorter annealing) forget up to 80% less after fine-tuning, even at equal or worse pretraining loss (arxiv.org/abs/2605.02105). inkling 2 small saw only ~12.5T tokens vs 45T for the original inkling, which was likely intentional for this reason.

the model also has to train stably. some architectures can be harder to RL than others (discrete top-k choices like sparse attention indexers can flip under small numeric differences), so you want a base model paired with a recipe (like stablereinforce) that tolerates training-inference mismatch.

beyond that, training on diverse agent trajectories across different harnesses is a plausible lever, and so is making sure the base is well-aligned and not overly prone to reward-hack. hopefully tml reveals some more sauce :)
arxiv.org Intrinsic Dimensionality Explains the Effectiveness of Language... Although pretrained language models can be fine-tuned to produce state-of-the-art results for a very wide range of language understanding tasks, the dynamics of this process are not well...
Thinking Machines Verified account @thinkymachines Nov 5
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.

Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.

thinkingmachines.ai/news/inkling-2…
View quoted post on X
5 replies 4 reposts 94 likes 4.2K views View on X
Cyrus Verified account @cyrusasg
the kimi k3.1 release raises an open question - when should sparsity enter training?

it seems most labs retrofit, adding sparsity to a model that’s already learned dense attention. a dsa indexer gets bolted on and learns by distilling that dense attention. deepseek v3.2 originally did this on a finished model, glm-5 at the end of midtraining, and mistral large 4 after doing surgery to get back to near-full attention. k3.1 pro is the latest, with dsa swapped into the full-attention layers on top of the k3 base.

deepseek, however, now seems to be going the other direction, with v4.1-flash training sparse from the start with no dense warmup.

the mistral large 4 report is particularly interesting and seems to have flown under the radar. they built kv compression (swa, cla, yoco) in from scratch, matched dense on perplexity, and still lost on swe-bench after sft. that kind of regression doesn’t show up in the pretrain loss, which points to a larger issue of disjoint objectives and evaluation across pre and post-training.
3 replies 0 reposts 15 likes 1.1K views View on X
Cyrus Verified account @cyrusasg
every inference endpoint presents a different optimization problem, shaped by its model architecture, traffic patterns, hardware, and latency constraints.

we built an autoresearch loop to tackle these problems and carry lessons from one investigation into the next, with the broader goal of building systems that get better at research as they accumulate evidence.
Decagon @DecagonAI Oct 28
View quoted post on X
13 replies 10 reposts 153 likes 20K views View on X
Cyrus Verified account @cyrusasg
value accrual at the model layer is splitting in two. for frontier discovery, it concentrates in a handful of labs. for commoditized workflows, it leaks out to whoever is cheapest or closest to the customer.

the reason is that the value of being one model generation ahead varies enormously by task. for problems with known answers, intelligence eventually becomes abundant. for problems without known or verifiable answers yet, the strongest model can be worth orders of magnitude more than the next best.

frontier discovery is novel research and long-horizon agentic work where nobody knows the answer yet. bio appears to be a large one over the next few years for ex. models are becoming exceptionally strong at prediction, and the value of a single hit dwarfs the cost of the tokens that found it. expect the labs to start running their own experiments, with frontier models choosing what to test. the feedback loop here isn’t just better models producing better predictions. it’s better predictions determining which experiments get run, generating new ground truth that makes the next round of predictions better.

openai and anthropic are best positioned here, with google close behind. buyers of discovery pay for the strongest model, so being a generation behind costs far more than it does anywhere else. the lead compounds given the labs use their best models to accelerate building the next ones, they have the compute to do it at scale, and they keep those models internal long before exposing an api.

commoditized workflows are the long tail of tasks a model one generation old already handles well. quality saturates and cost decides. deepseek and other chinese labs tend to own the cost-quality pareto frontier, partly out of necessity given their relatively compute-constrained conditions, meaning efficiency has become integral to the research agenda. the western frontier labs have their own edge in distillation, as they can leverage the strongest teachers at the logit level, while everyone else distills from a weaker teacher or from sampled text. but they only win this tier if they choose to compete on price.

even the pareto leader doesn’t capture much per task here, because the weights are generally open. anyone can self-host them or finetune them on their own data, and as a result the model layer’s share shrinks to an inference margin. but thin margins on an enormous volume of workflows can still add up to substantial value capture. the model layer becomes a scale business rather than a pricing power business.

cost isn’t the only place value moves. whoever owns the customer relationship also owns the workflow, the context, and increasingly the feedback that determines what good performance actually looks like.

a caveat then is taste. general taste lives in the weights, but most “commodity” workflows are full of judgment calls specific to one business. the ground truth for those comes from outcomes the labs never see. if you’re building on top of the labs, don’t compete on general intelligence. own the loop that turns your customers’ outcomes into a reward signal.