Twitxy Saved

Luis

Verified account This account is verified. Learn more

@lusxvr

ELLIS PhD Student @TU_Muenchen | prev: @huggingface @DLR_de
lusxvr.github.io Joined January 2015
566 Following 1.1K Followers 335 Posts

Latest posts 5

Pinned post

Luis Verified account @lusxvr
Agents are evaluated with ever more detailed metrics, but why don't we study how their own configuration shapes performance?

We study this on 4 scientific tasks, by varying 5 parts of the agents configuration across 18,000+ runs.

Here are my favorite findings:
1 replies 4 reposts 15 likes 901 views View on X
Luis Verified account @lusxvr
Paper Spotlight: Can a team of AI agents do research from idea to accepted paper?

"ScientistTwo" got 107 problems taken from ICLR, ICML and NeurIPS papers:
-> Beat the human state of the art on 86 (80.4%)
-> Average gain: 25.2%
-> Cost per paper: ~$3,800
2 replies 2 reposts 29 likes 1.8K views View on X
Luis Verified account @lusxvr
Paper Spotlight: Do more AI agents always make a better system?

-> relative performance of multiple agents compared to single agents ranges from +80.8% to -70.0%, depending on the architecture-task alignment

"Agent effectiveness depends on alignment between coordination and task structure, and mismatched coordination degrades the performance."
4 replies 3 reposts 30 likes 1.1K views View on X
Luis Verified account @lusxvr
Paper Spotlight: Do AI agents notice what is right in front of them?

If the full solution to a task is injected into the agent's environment, e.g. a solution.​sh file in the working folder on Terminal-Bench,

-> Agents find it in about 80% of runs
-> They use it in only 37-50% of runs