Twitxy Saved

Sebastian Gehrmann

@sebgehr

Head of Responsible AI, CTO office @Bloomberg.
Formerly LLM evals @ Google / PhD @ Harvard.

views my own
New York City sebastiangehrmann.github.io Joined November 2013
2K Following 6.8K Followers 1.7K Posts

Latest posts 5

Sebastian Gehrmann @sebgehr
This is cool work but I am sad that we keep reinventing the wheel. Most of these trends are not emerging and have been studied *extensively*. See these surveys:
- arxiv.org/pdf/2006.14799
- jair.org/index.php/jair… (I am author)

313 citations and only 15 from before 2023 :(
Arvind Narayanan Verified account @random_walker Oct 20
As LLMs evolved into agents, earlier best practices no longer serve us well. This new research field needs a field guide, so we wrote one. If you're new to agent evals, please take a look. Seasoned researchers and those interested in steering the community will also find many useful observations. sage.cs.princeton.edu/trends/
View quoted post on X
0 replies 2 reposts 34 likes 3.4K views View on X
Sebastian Gehrmann @sebgehr
For domain-specific AI Systems, we need to trust in repeatable processes and cannot rely on LLMs to get that right. But hard-coding everything leads to context bloat.

Skills over MCP + progressive disclosure solve this!

See our engineering blog: bloomberg.com/company/storie…
www.bloomberg.com Skills over MCP: How Bloomberg Helped Standardize Agent Skills at Scale | Bloomberg LP Learn how Bloomberg engineers helped standardize 'Skills over MCP' to help scale agentic AI.