Twitxy Saved

Yiding Jiang

Verified account This account is verified. Learn more

@yidingjiang

Research @GoogleDeepMind | Prev: PhD @mldcmu, AI resident @GoogleAI, BS @Berkeley_EECS. Trying to understand stuff.
yidingjiang.github.io Joined December 2015
664 Following 2.8K Followers 419 Posts

Latest posts 5

Pinned post

Yiding Jiang Verified account @yidingjiang
Information theory often gives unintuitive conclusions when it comes to data. Many of these inconsistencies can be resolved elegantly if we limit the amount of computation the observers can use. Very happy to finally introduce our work on epiplexity! 1/馃У
Marc Finzi Verified account @m_finzi Jan 7
1/馃У We are very excited to release our new paper! From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
arxiv.org/abs/2601.03220
with amazing team @ShikaiQiu @yidingjiang @Pavel_Izmailov @zicokolter @andrewgwils
View quoted post on X
6 replies 16 reposts 72 likes 13K views View on X
Yiding Jiang Verified account @yidingjiang
10M 馃槷 looking forward to the forest 馃尦
Behnam Neyshabur Verified account @bneyshabur Oct 26
Introducing Fern, Mirendil鈥檚 first model: pretrained from scratch, with up to 10M tokens of context!

That鈥檚 ~0.5M lines of code (roughly Megatron-LM scale) or days of an agent鈥檚 work, all in context at once.

Its first version, Fern 0.1, had a short round of post-training 馃殺
View quoted post on X
Yiding Jiang Verified account @yidingjiang
This kind of result would be unimaginable one year ago. Congrats to the team!
Behnam Neyshabur Verified account @bneyshabur Oct 21
We formally announced Mirendil less than 4 months ago. Today, I'm excited to share an important milestone for:

Pretraining at scale without a pretraining team.

Mirendil's mission is to solve Recursive Self-Improvement (RSI) and use it to democratize frontier AI development, ultimately accelerating science and technology. At Mirendil, when we think about RSI, we think holistically about the company as an intelligent entity that self-improves toward its goal. From that perspective, the most powerful notion of acceleration is reducing the time and resources (number of human experts, compute, and other costs) needed to achieve ambitious goals.

Given that pretraining is perhaps the most important yet most challenging part of frontier AI development, we decided to run an ambitious controlled experiment: Starting fully from scratch, can we significantly reduce the time, number of human experts, and compute resources needed to go all the way to successfully pretraining a strong coding model at scale?

Mirendil-Base is the result of this experiment. Our first pretrained model was built by one researcher in 3.5 months, with less than $10M in compute. This includes all the research and small-scale experiments it took to get there, as well as the final training run.

The recipe holds up as the models get bigger. For reference, Mirendil-Base is in the 100B-500B parameter range, Mirendil-Base-Large in the 500B-2T range, and Mirendil-Base-XL in the 2T-10T range.

We're beginning to see a world where a single researcher, empowered by AI, can take on work that once required an entire research organization. As these systems improve, the bottleneck shifts from the size of a team to the ambition and judgment of the people directing them. This has profound implications for who gets to build AI and, ultimately, who gets to push the frontiers of science and technology forward.

We鈥檒l share independent evaluations, post-training benchmarks and API-access sooner than you think so stay tuned!
View quoted post on X