Twitxy Saved

Sam Dare

Verified account This account is verified. Learn more

@DistStateAndMe

Founder / Member of Non Technical Staff @covenant_ai ( @tplr_ai : @basilic_ai : @grail_ai )
covenant.ai Joined April 2014
3K Following 4.5K Followers 10K Posts

Latest posts 5

Pinned post

Sam Dare Verified account @DistStateAndMe
A small step for mankind, a massive leap for decentralised training... for agency.

In the space of 9 months, @tplr_ai went from 1.2B -> 72B.

It's never been easy, and has broken everyone on the team multiple times. But I speak for all of us when I say it is the most rewarding thing we have ever done.

We have a fraction of the resources. We don't have the PhDs. But Bittensor shows you it doesn't matter. Innovation happens at the edge. We innovate through scarcity.

The ones who rewrite the rules are never the ones with the most. They're the ones who refuse to accept the limits they were handed.

Bittensor is prophecy. Subnets (@covenant_ai and others) are the tools through which that prophecy is manifested.

Next stop: TRILLIONS.
templar @tplr_ai Mar 10
We just completed the largest decentralised LLM pre-training run in history: Covenant-72B. Permissionless, on Bittensor subnet 3.

72B parameters. ~1.1T tokens. Commodity internet. No centralized cluster. No whitelist. Anyone with GPUs could join or leave freely.

1/n
View quoted post on X
38 replies 33 reposts 271 likes 76K views View on X
Sam Dare Verified account @DistStateAndMe
The ability to terraform intelligence over a globally distributed swarm has always been the mission.

We present a 30B MOE run,

The Internet is the center .

Higher 🚀
templar @tplr_ai 2d
Most of today's leading open models are mixture-of-experts. Crucible, Templar's pre-training platform, has finished its first MoE test run.

The model follows the Qwen3-30B-A3B architecture from @Alibaba_Qwen, with 30B parameters and 128 experts, 8 of them active per token. On 32 A100 GPUs, two replicas of four pipeline stages averaged about 200K tokens per second over more than 50B tokens.

Watch our runs live at tplr.ai/dashboard
View quoted post on X
1 replies 4 reposts 25 likes 1.5K views View on X
Sam Dare Verified account @DistStateAndMe
We are small, but managed to contribute to foundational algorithm for modern RL.

We will stay the course , doubling down on the mission . Train / serve everywhere .

Unite the silos.

The internet is the datacenter
templar @tplr_ai Nov 3
PULSE (also called Delta weight sync) has become one of the most important ideas in RL infrastructure this year. Our paper, posted on February 3, explained why it loses nothing. At RL learning rates, about 99% of each step's weight updates disappear in the BF16 cast, so the inference workers never see them.

Since then, @huggingface has built Delta Weight Sync for TRL and vLLM on that argument. @FireworksAI_HQ cites PULSE, and its ARCv3 compressor now makes the updates even smaller.

This week @humansand reported pausing inference for only about 220 milliseconds per update on Blackwell GPUs.

PULSE has now been accepted at AXIOM, a @NeurIPSConf 2026 workshop.

arxiv.org/abs/2602.03839
View quoted post on X
2 replies 1 reposts 24 likes 1.3K views View on X
Sam Dare Verified account @DistStateAndMe
We live in an alternate reality, and i am loving it . Accelerate 🚀
Rikiya Takehi @rikiyatakehi Nov 2
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.

We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weight-sp…
View quoted post on X