Twitxy Saved

Joshua Gu

@astrogu_

PhD student @MIT_CSAIL @MIT, Recursive Research๐Ÿ‘จโ€๐Ÿ’ป | Previous: @LMCache, @tensormesh, undergrad @UChicago
Chicago, IL Joined December 2023
286 Following 341 Followers 71 Posts

Latest posts 5

Pinned post

Joshua Gu @astrogu_
Recent agentic systems (Claude Code, Codex, RLM, etc.) push context out of the prompt and into the environment (e.g., as files). This helps them maintain long-term knowledge about their goals and functionality.

๐Ÿšจ While this is a good idea, we show a surprising result: systems that use external environments like this perform much better when given a small, fixed-size, in-context, agent-managed cache that "๐˜ฑ๐˜ฆ๐˜ฆ๐˜ฌ๐˜ด ๐˜ช๐˜ฏ๐˜ต๐˜ฐ" these environments.

๐Ÿš€ Our paper, ๐—ฃ๐—˜๐—˜๐—ž: ๐™– ๐™จ๐™ฎ๐™จ๐™ฉ๐™š๐™ข ๐™›๐™ค๐™ง ๐™—๐™ช๐™ž๐™ก๐™™๐™ž๐™ฃ๐™œ ๐™–๐™ฃ๐™™ ๐™ข๐™–๐™ž๐™ฃ๐™ฉ๐™–๐™ž๐™ฃ๐™ž๐™ฃ๐™œ ๐—ฎ๐—ป ๐—ผ๐—ฟ๐—ถ๐—ฒ๐—ป๐˜๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—ฐ๐—ฎ๐—ฐ๐—ต๐—ฒ ๐™›๐™ค๐™ง ๐™‡๐™‡๐™ˆ ๐™–๐™œ๐™š๐™ฃ๐™ฉ๐™จ, introduces this idea.

Compared with strong baselines, including RAG, Compaction Agents, and SOTA prompt-learning frameworks, PEEK dominates the costโ€“quality Pareto frontier: achieving +6.3โ€“34.0% in quality, with fewer iterations and lower cost.

Paper: arxiv.org/abs/2605.19932
GitHub: github.com/zhuohangu/peek

More in the thread below! (1/N)
17 replies 40 reposts 365 likes 117K views View on X
Joshua Gu @astrogu_
2 years ago in LLMSteer we picked which layers and heads to steer by brute-force profiling (w/o fine-tuning!). the follow-up I always wanted to work on was letting the LM figure out the edits itself. Now astraโ€™s doing it straight on the weights. congrats @rikiyatakehi!
Rikiya Takehi @rikiyatakehi Nov 2
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.

We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! ๐Ÿงต
Blog: rikiyat.github.io/blog/weight-spโ€ฆ
View quoted post on X
Joshua Gu @astrogu_
Canโ€™t believe I was the last to find out ๐Ÿ™‚โ€โ†”๏ธ Loved every moment working on this project, first of many to come!

Huge thanks to my collaborator @qizhengz_alex and advisors @lateinteraction @samrmadden
Qizheng Zhang Verified account @qizhengz_alex Sep 24
PEEK has been accepted to NeurIPS 2026! ๐ŸŽ‰

One thing I learned from this project: even with ever-larger context windows, language models and coding agents can still get lost in large files and repos. Thatโ€™s why I love the idea of a โ€œContext Map as an Orientation Cacheโ€ --- more context alone doesnโ€™t mean better navigation!

Huge credit to @astrogu_ and the mentors!
View quoted post on X
0 replies 0 reposts 14 likes 637 views View on X
Joshua Gu @astrogu_
Iโ€™ll be at @VLDBconf in Boston this week. Come say hi and swing by our demo, talk, or poster!
Matthew Russo Verified account @RussoMatthew Aug 30
Letting agents naively query your enterprise data lake can be error-prone and expensive. What if we could structure their execution in a way that was steerable and easy to optimize for cost-efficiency? This week Iโ€™ll be presenting two research papers at VLDB which work towards this goal.

#Carnot: vldb.org/pvldb/vol19/p4โ€ฆ
#Abacus: vldb.org/pvldb/vol19/p1โ€ฆ

#Carnot is a system for interpretable, interactive, and optimized execution of agentic search queries. Carnot lets business users issue queries in natural language, compiles them to an optimized computation graph, and executes them. A notebook interface gives users control over the agentโ€™s query, enabling them to inspect intermediate outputs while steering the agent back to the correct computation when it makes a mistake. Finally, Carnot helps users manage the cost of their queries by predicting the cost of each operation and opting for plans which can satisfy a userโ€™s specified budget.

#Abacus is an optimizer for semantic query processing which we built and shipped in Palimpzest. Abacus optimizes semantic queries (with or without relational operations) across all three dimensions of quality, cost, and latency. The optimizer was designed to be extensible and scalable, leveraging a multi-armed bandit search algorithm to find the best operators for each query plan and optimization objective. In our evaluation, Abacus' plans achieved ~7%-40% better quality while being 10.8x cheaper and 3.4x faster (on average) than the next best system(s).

This work is the product of a great research team from MITโ€™s Data Systems Group and OASYS Lab: Yash Agarwal, @tianyu_li_, @astrogu_, @chunwei_l, Siva Sudhir, @gerarvita, @MikeCafarella, @lateinteraction, @tim_kraska, and @samrmadden

If youโ€™re at VLDB this week and curious about our current (and future) work, please swing by our demo, research talk, and poster sessions!
View quoted post on X
Joshua Gu @astrogu_
had a great chat with Yuzhen about this work during my Stanford visit two weeks ago. congrats on the release!
Yuzhen Mao Verified account @Mao_Yuzhen Jun 10
What happens when multi-agent systems stop relying on a central โ€œcontrollerโ€ agent? Can agents coordinate by sharing results directly with each other?

Introducing Decentralized Language Models (DeLM): we let agents coordinate asynchronously through a shared context. Agents claim tasks from a queue and write back compact, verified results as they finish, making progress visible to all workers without requiring a main agent to merge, filter, and rebroadcast it.

New paper with @azaliamirh!
View quoted post on X