Twitxy
Saved
@XLangNLP's profile picture
View on X

XLANG NLP Lab

Verified account This account is verified. Learn more

@XLangNLP

developing embodied ai agents that empower users to use language to interact with digital and physical environments to carry out real-world tasks.
xlang.ai Joined July 2023
36 Following 1.7K Followers 143 Posts

Latest posts 5

Pinned post

XLANG NLP Lab Verified account @XLangNLP
Two years ago, we built OSWorld 1.0 — the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved?

Not even close.

🚀Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks.

What's new:
🎯 108 real-world workflows, each ~1.6 hours ⏱️ for a skilled human
⚙️ ~318 tool calls/task vs. ~30 in OSWorld 1.0
🌍 Grounded in authentic artifacts & stateful user profiles
⚡ Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more

📊 Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use.

🏠 Homepage: osworld-v2.xlang.ai
📄 Paper: github.com/xlang-ai/OSWor…
💻 Code: github.com/xlang-ai/OSWor…
🤗 Dataset: huggingface.co/datasets/xlang…

🧵 [1/8]
0:23
16 replies 109 reposts 454 likes 174K views View on X
XLANG NLP Lab Verified account @XLangNLP
Congrats the Claude team on the big jump on OSWorld!

Worth flagging this 70.6% is partial-credit scoring — under strict binary task-level success (OSWorld 1.0-style), Opus 5 is still only ~30%.

We're working on improving task quality so task-level binary success becomes more robust and reliable even on hard tasks too.
Claude Verified account @claudeai Jul 24
On several coding and knowledge work evaluations, Opus 5 is the new state-of-the-art:
View quoted post on X
1 replies 2 reposts 20 likes 4.1K views View on X
XLANG NLP Lab Verified account @XLangNLP
Check out our latest open-source project, OpenCUA, for Computer Use Agents (CUAs)!

Find the code, annotation tool, the largest CUA dataset, scalable training recipe, and state-of-the-art model on OSWorld at: opencua.xlang.ai.
Xinyuan Wang @xywang626 Aug 15, 2025
We are super excited to release OpenCUA — the first from 0 to 1 computer-use agent foundation model framework and open-source SOTA model OpenCUA-32B, matching top proprietary models on OSWorld-Verified, with full infrastructure and data.

🔗 [Paper] arxiv.org/abs/2508.09123
📌 [Website] opencua.xlang.ai
🤖 [Models] huggingface.co/xlangai/OpenCU…
📊[Data] huggingface.co/datasets/xlang…
💻 [Code] github.com/xlang-ai/OpenC…

🌟 OpenCUA — comprehensive open-source framework for computer-use agents, including:
📊 AgentNet — first large-scale CUA dataset (3 systems, 200+ apps & sites, 22.6K trajectories)
🏆 OpenCUA model — open-source SOTA on OSWorld-Verified (34.8% avg success, outperforms OpenAI CUA)
🖥 AgentNetTool — cross-system computer-use task annotation tool
🏁 AgentNetBench — offline CUA benchmark for fast, reproducible evaluation

💡 Why OpenCUA?
Proprietary CUAs like Claude or OpenAI CUA are impressive🤯 — but there’s no large-scale open desktop agent dataset or transparent pipeline. OpenCUA changes that by offering the full open-source stack 🛠: scalable cross-system data collection, effective data formulation, model training strategy, and reproducible evaluation — powering top open-source models including OpenCUA-7B and OpenCUA-32B that excel in GUI planning & grounding.

Details of OpenCUA framework👇
View quoted post on X
1 replies 5 reposts 23 likes 2.5K views View on X