Twitxy Saved

Stas Bekman

Verified account This account is verified. Learn more

@StasBekman

Toolmaker. Software creator, optimizer and harmonizer.

Makes ML systems work and fly @ Snowflake.
Nanaimo, BC, Canada stasosphere.com/machine-learni… Joined November 2018
311 Following 12K Followers 3.1K Posts

Latest posts 5

Stas Bekman Verified account @StasBekman
"If you are interested in human-level AI, don't work on LLMs." -- Yann LeCun
K (Khashayar) Mansouri Verified account @k_mansourizadeh Sep 16
Professor @ylecun made some crucial points recently. In my opinion, the most important one is: "If you are interested in human-level AI, don't work on LLMs."

Virtually every word in this presentation drives our work on "Forge" and "MALWIM" (Multi-Stage Agentic Loops With Inhomogeneous Models). We believe in an AI future that is specialized, democratized, affordable, and unique to each use case and company. We do not believe in a centralized AI landscape dominated by just a few massive players. The reason is clear: once you make models compact, efficient, and affordable, companies switch from being mere "wrappers" to actual model creators within their core application layer.

Beyond that strategic purpose, LLMs deeply lack the capabilities required for human-level AI, such as internal long-term memory, the ability to predict actions and consequences, and a holistic, global understanding of a topic.

The mainstream world may still be figuring out LLMs, riding the hype wave, and catching up to what visionaries like @ylecun view as a dead-end architectural path. Mainstream attention might not shift for another 24 months. However, once you see the reality, you cannot unsee it: no AGI, no ASI, and no major future breakthroughs can be achieved through sole-LLM architectures and basic agent designs.

We are currently training our first Forge/MALWIM model and agentic system, specifically designed for coding. Over the coming months, we will release benchmarks, results, and potentially a limited-access beta launch.
View quoted post on X
3 replies 0 reposts 5 likes 1.1K views View on X
Stas Bekman Verified account @StasBekman
Bummer, another correctness fix in Zero1/2 @DeepSpeedAI
github.com/deepspeedai/De…

please upgrade to deepspeed>=0.19.9

Thanks to Sean Bell for finding and fixing the issue.

I wonder how is it possible all these bugs were there for years and getting discovered only now.
github.com ZeRO-1/2: preserve gradients across request boundaries until step by stas00 · Pull Request #8949 ·... Additionally to #8749 another precision issue Porting this from @seanbell's Snowflake-AI-Research/Arctic-Platform#371 An accumulation boundary is not necessarily an optimizer step. A caller...
4 replies 3 reposts 18 likes 2.1K views View on X
Stas Bekman Verified account @StasBekman
I finally found a way for at least gpt-6.1-sol not to drop the ball and work autonomously 24/7 for many weeks now.

Basically the trick is to make the agent remind itself what's important at the end of each of its replies.

I also made it make a special header which stands out from the sea of self-talk messages in codex, so now it's easy to see what I should read.

And I made it report status every 30min, which ensures the reminding happens always.

The epilogue is:

I must not stop unless the operator explicitly told me to stop. I must always reply with the proper header from the rules

Attached is an example of a response, scratched the part that is irrelevant to what I'm trying to show.
23 replies 8 reposts 128 likes 9.2K views View on X
Stas Bekman Verified account @StasBekman
@Snowflake is Introducing Cortex Training, this is the project my AI Research team has been working on since the beginning of this year

snowflake.com/en/blog/cortex…

Cortex Training is Snowflake’s serverless offering for building and deploying customized open-weight models. AI teams retain fine-grained control over SFT, RL and custom training loops, while Snowflake manages the GPU infrastructure so developers can focus on improving models rather than operating clusters.