Pinned post
AI doesn't crash; it's confidently wrong.
So who verifies the AI?
Sign up to have your AI tested. testmuai.com
So who verifies the AI?
Sign up to have your AI tested. testmuai.com
2:32
Latest posts 5
Red-Teaming ≠ Agent Assurance: Cisco AI Defense + Splunk: What They Check, What’s Missing x.com/i/article/2120…
Claude Fable 5.2 passed 9 more tasks per 100 than Opus 5.5 on Zapier AutomationBench. It cost 3.2x x.com/i/article/2120…
The quiet hero here is Lean. An agent swarm wrote it, and Lean won’t accept a proof unless it actually holds. Most AI-written apps and agents ship with nothing like that. That’s the gap we’re closing at @testmuai .
Most think vibecoding at scale is slop. Here’s a counterexample.
Today we release a 16M-line formalization of the entire Stacks Project, critical infra underlying modern math, with optimized code quality and its highly interconnected structure preserved.
A mathematical city 🌆View quoted post on X
@nikesharora is right. AI can find a security hole in seconds. But every fix still has to be tested so it stops the attack without breaking the app. That takes days.
So the fix waits, and the hole stays open. The slow part of security is now testing, not finding.
So the fix waits, and the hole stays open. The slow part of security is now testing, not finding.
What people don't know is that - frontier LLMs are not "defending customers”, they are exposing vulnerabilities - defense means - no false positives, comprehensive coverage, and a cost efficient solution. Any defense involves, sensors, inline inspection and enforcment, this can only come with a collaborative approach.View quoted post on X
It's time for frontier LLMs to take a responsible approach to Cyber. I have been watching them consistently tout their abilities on finding vulnerabilities and daisy chaining them to be able to attack enterprises. I have also recently learnt about their efforts to create synthetic data to enhance their capabilities. I am all for it. But don't use this as a tactic to require each enterprise to scan for vulnerabilities again.
The narrative needs to change - we already know that we are in a cycle where all enterprises need to remediate their vulnerabilities. It is important to understand:
1. Vulnerability discovery is existential, because bad actors will - patching it is even more so. Expending efforts at more vulnerability discovery needs to be balanced with a solve.
2. LLMs are capable of recommending patches, but before a patch is developed, and deployed, it needs to be tested in its environment to ensure continued functioning of the application, and even now the solve has high false positives. No enterprise can do it all at speed.
3. Despite their enhanced capability, there is still significant difference in what vulnerabiities are found across all their Frontier LLM's, so customers will need to test across a multi-model harness. One LLM isn't the solution.
4. Most organizations do not have capacity to patch at the speed needed, especially the smaller ones.
5. We don't talk about operational technology enough, most critical infrastructure relies on OT and we need a solve for them in addition to Open Source.
The most obvious solution is virtual patching. This is a time tested technology that is currently available with cyber vendors for endpoints, WAF, cloud, network use cases. It also is not comprehensive enough currently, we need to scale this fast. We need a vulnerability clearing house and fast signature development for all these vulnerabilites to respond at speed across our industry.
