Twitxy Saved

Shane Gu

Verified account This account is verified. Learn more

@shaneguML

Sr. Staff RS @GoogleDeepMind. 🇯🇵-born 🇨🇳🇨🇦. Gemini 1.5+, Omni Thinking, GPT-4 @OpenAI, GoogBrain Robotics (JP: @shanegJP). Personal opinions.
Mountain View, CA scholar.google.com/citations?user… Joined June 2016
2.3K Following 52K Followers 4.4K Posts

Latest posts 5

Pinned post

Shane Gu Verified account @shaneguML
I had the pleasure of catching up with Geoff Hinton yesterday to celebrate his birthday. As we took a long, winding walk along the coast, I was reminded of: not all tokens are born equal.

Just as his insights and words from a decade ago still trigger new chains of thought for me, yesterday’s conversation was exceptionally fruitful. We spent some time discussing Tim Lillicrap, my current manager and co-lead of Gemini thinking, whose research on random matrix feedback from ten years ago actually inspired my very first paper with Ilya Sutskever. Geoff offered his perspective on exactly why that method worked, dissecting the phenomenon with precise intuitions grounded in mathematics.

Our discussion ranged from the technical to the philosophical. He remains as intellectually open-minded as ever, still imagining for true novel breakthroughs that go beyond the current paradigms of backpropagation. Our last (brief) chat was shortly after his foward-forward algorithm in late 2022. We also explored global perspectives, touching on the dynamics between China, Japan, and the West. We discussed how the philosophical roots of Asian cultures might be better suited for the societal shifts of a post-AGI world, and what the West could learn from the East to ensure that the future is positive.

On a personal note, I was touched by his genuine concern for the next generation. When I mentioned my daughter, he agreed that raising children is the ultimate way to reshape one’s priorities--a grounding force that liberates you from immediate 'capitalist FOMO' and refocuses you on the long term. His words carry the weight of years of high-quality processing; they aren't just outputs of raw intelligence, but of deep, sincere care for humanity. It is clear he cares about far more than any single technology or nation. His reasoning and stories serve as a powerful reminder to understand and explore the broader world—proof that a vast perspective is what shapes greater wisdom and enhances our humanity.

I attended NeurIPS 2024 hoping to congratulate him on his Nobel Prize but missed the chance. I am so glad I was finally able to do so this year on his birthday. And finally, he left me with a few more important predictive tokens: 'Google and Gemini will win' :)"
15 replies 31 reposts 555 likes 125K views View on X
Shane Gu Verified account @shaneguML
A frontier model that masters symbols, space and time is all you need. It's been pleasure collaborating and working with Gaming and 3D efforts in Gemini. More to come!
koray kavukcuoglu Verified account @koraykv 2d
To see how its 3D spatial understanding, multimodal reasoning, and coding come together, the team gave Neon a raw Smithsonian 3D scan of a woolly mammoth. Neon segmented the scan into individual bones, arranged and rigged them anatomically into a full 3D skeleton, and generated this museum walkthrough of the build.
View quoted post on X
7 replies 2 reposts 48 likes 4.4K views View on X
Shane Gu Verified account @shaneguML
Excited to see Gemini 4 Argon finally released! Over the past few weeks, my team collaborated with another to advance Gemini’s capabilities in game dev. Game dev pushes beyond symbolic intelligence to eval rich spatio-temporal reasoning, a crucial step toward physical AGI.
Demis Hassabis Verified account @demishassabis Oct 13
Gemini 4 Argon is amazing for knowledge work and coding - and it’s super fun to build with! We asked it to create this wizard chess game and it's so cool to see Argon's physics understanding & 3D modelling capabilities, makes me nostalgic for Battlechess on the Amiga - fun times!

Argon is rolling out today to Google Ultra and Pro subscribers in the @GeminiApp and Search, and Ultra subscribers in @antigravity! More info & full availability in the blog: blog.google/innovation-and…
View quoted post on X
7 replies 7 reposts 111 likes 7.8K views View on X
Shane Gu Verified account @shaneguML
I felt a similar despair 4-5 years ago as I was still partially working on robotics. After co-authoring "LLMs are Zero-Shot Reasoners" and "LLMs Can Self-Improve" in 2022, it became clear that digital/symbolic AGI had to precede, and would ultimately accelerate, physical AI. While video models are powerful zero-shot reasoners (video-zero-shot.github.io), LLMs remain the indispensable core in every aspect of physical AGI development. Over the past year, I've actively collaborated across Gemini, Omni, and generative media to test my hypotheses regarding physical AGI. The pieces are finally coming together. It is clear that frontier LLM labs, alongside the rest of the robotics community, are uniquely equipped to drive this next transition in embodied AI.
video-zero-shot.github.io Video models are zero-shot learners and reasoners Video models like Veo 3 are on a path to become vision foundation models.
Lerrel Pinto Verified account @LerrelPinto Sep 12
I’m sensing deep despair in academics over the past week. Astra, Fable, Muse are zero-shotting benchmarks in robotics & world models.

Uneasy pill to swallow, but this is what step jumps in progress looks like.
View quoted post on X
13 replies 36 reposts 381 likes 104K views View on X