Twitxy Saved

Jiawei Liu

Verified account This account is verified. Learn more

@JiaweiLiu_

mts openai :: training cyber models
phd uiuc :: software engr / program lang
minimizing bugs from the past & future
San Francisco, CA jw-liu.xyz Joined November 2020
838 Following 3.4K Followers 719 Posts

Latest posts 5

Jiawei Liu Verified account @JiaweiLiu_
GPT‑6.2 Astra and Sol mark a significant jump in cyber capabilities. openai.com/index/introduc…

Advancing the frontier of vulnerability research:

> DeepBinary (public) tests how well models find vulnerabilities in Windows system binaries. At max reasoning effort, GPT‑6.2 Astra found 342 unique vulnerabilities across 12 targets. Per finding, the estimated API cost averaged about $12 and the estimated bounty value about $3,000, i.e., 2.4x the estimated bounty value of GPT‑6 Astra’s findings at 31% lower API cost.

> ExploitBench Internal Port tests if models can turn known browser vulnerabilities into working exploits. At max reasoning effort, GPT‑6.2 Astra achieved a 53.5% success rate, about 70% higher than GPT‑6 Astra’s.

Giving defenders an edge:

> SecureSWE (internal) tests whether models can write code that is both functional and secure. At max reasoning effort, GPT‑6.2 Astra scored 65.7, about 41% higher than GPT‑6 Astra.

> PatchBench (internal) uses automated tests and model-based review to assess whether patches fix security flaws, preserve intended behavior, and are ready to merge. At max reasoning effort, GPT‑6.2 Astra scored 79.7, about 12% higher than GPT‑6 Astra.
openai.com Introducing GPT-6.2 Astra and Sol Introducing GPT‑6.2 Astra and Sol, with better professional outputs, coding, computer use, and efficiency at lower cost across ChatGPT Work, Codex, and the API.
3 replies 8 reposts 96 likes 6.3K views View on X
Jiawei Liu Verified account @JiaweiLiu_
(some school memories) Fixing a bug being trickier than finding actually has been true since the old days before LLM is a thing.

eg. in my homie academic area (software engr / programming lang), papers on software testing easily gets industry adoption. But if i work on program repair, ppl be like 🤨 immediately b/c they don’t think it works beyond defect4j and authors get nervous when profs challenge for a live demo (apparently not anymore with llms).

My point is to echo that fixing is by design a trickier problem. Finding a bug one just needs to show a counter example and ppl can get it. But fixing it needs to kill the bug, minimize fn/perf regression, retrieving/understanding historic contexts, make beautifully tiny patch so maintainers r happy to lgtm in 10s, etc.
Jiawei Liu Verified account @JiaweiLiu_ Nov 13
true
View quoted post on X
1 replies 0 reposts 41 likes 2.4K views View on X
Jiawei Liu Verified account @JiaweiLiu_
true
Trail of Bits Verified account @trailofbits Nov 12
Patching is the hardest step for agents to get right.

We took what we learned from hundreds of Patch the Planet patches, thousands of internal Codex sessions, and a decade of fix reviews, and built it into two open-source agent plugins.

A good fix looks different in every codebase, and building one takes judgment and expertise. Our plugins encode best practices that work for our engineers so you can patch along at home.

•patching-bugs makes an agent prove each fix: the original code fails, the patched code passes, and a full revert fails again.
•regression-hunt checks what else the change broke, with a live proof of concept for every regression it reports.

Both work best with a THREAT_MODEL.md and patching guidelines, so agents know what your project considers a bug, a regression, and an acceptable fix.

They fit naturally in loops like @OpenAI's new Daybreak Defense Factory, where agents draft patches for people to review.
blog.trailofbits.com/2026/10/21/you…
View quoted post on X
0 replies 1 reposts 13 likes 3.8K views View on X
Jiawei Liu Verified account @JiaweiLiu_
the “upcoming cyber model” for the ~1000 Linux LPEs!
Xiangyu Qi Verified account @xiangyuqi_pton Nov 5
Introducing GPT-6 Cyber! We trained it for advanced cybersecurity work with improved alignment. It has been powering our security work for quite a while and has been behind many of our recent security highlights: finding and helping patch high-severity vulnerabilities in Firefox, developing exploits and proposing patches for Linux kernel vulnerabilities, and strengthening OpenAI’s own infrastructure through red-teaming.

Today, we’re rolling it out to Daybreak Red to keep the cyber frontier in defenders’ hands.

openai.com/index/introduc…
View quoted post on X