Trust

Anthropic continues to abstract away any notion of hands-on control over their codebase.

How reassuring is it for world leaders, tech companies, or to the average person, that some of the most consequential software in history is not even read by a human?

What is even the point of guardrails and AI safety models in a world where AI giants delegate their implementation to yet more AI? What does human-in-the-cockpit achieve when no one understands how the engine works? How many catastrophes have to happen before we stop sci-fi LARPing and come back to a sane accountability mindset?

When Claude is overtrained on audit contests and starts lawyering up... we've created a monster

We all need to take a moment and give our appreciation to the pioneers of decentralized security audits. I personally owe much of my success to the opportunities they've created. In the face of competition it continued to stay faithful to its policies and treated researchers fairly, even if it meant losing business.

It is true that in recent times C4 has become a shadow of what it once was, for various reasons. But I'll remember them for the epic competitions, intellectual judgment discussions and friends made on the way.

On the business side, this is part of the phase-out of crowd audits, which are no longer feasible to run with AI submissions. For years the gig has stopped being profitable and used for customer acquisition and upsell potential, but now it's official. Security is already converging to multiple AI passes followed by A-tier team audit(s) for finding outliers, and a bounty program as last defense.

So-long C4. You will be missed. 🐺

The ironic thing is that this will sound to many like old people come across when irrationally fearing technology. But this time the "technophobes" are right, and it's just a matter of how many AI catastrophes need to happen until the rest understand it.

Here's the core truth accelerationists don't want people to understand: every non-AI process running on your computer can, with some amount of effort, be explained by a computer scientist. Once AI is involved, the "why" something works is gone, and all you're left with is a pattern of processing that seems right on the surface. But without understanding the piping, by definition you cannot make any positive statement about its correctness.

Any time humans or computers act based on LLM output it carries an unbounded leak risk. To be clear this is not going away in the next Claude or ChatGPT release, it's completely structural and baked into the tech. Every user needs to decide whether the risk/reward tradeoff is worth it. The problem starts when (a) users are kept blind about these risks and (b) powerful players make the opt-in decision for everyone.

That is not to say that LLMs are not incredibly useful. Unfortunately (not particularly smart) people keep equating the arguments "LLMs are unreliable" and "LLMs are useless", this way they can cancel you and lets them keep chasing unrealistic world views.

I seriously hope I'm wrong about this, but we're going to see more and more "rm -rf /" tier screw ups in the coming years. Anyone reading tech news is already seeing this acceleration. But what's even more worrying than the computational leak is the human processing leak. What happens when elections are decided based on wrong, or manipulated premises? Can't say I'm waiting to found out. Humanity had an arguably good thing going before this, it's too bad financial greed (stock has to go up right?) is going to send us crashing into walls without as much as a seat belt.

CT focusing on the low-hanging fruit takes -
"How could KelpDAO be this negligent to configure 1/1 DVN?"
"Why is LZ supporting such unsafe config?"
"How did AAVE risk management team not flag this setup?"

While valid, these are the wrong questions to be asking.

As an industry after each exploit we always seem to point fingers at the most easy to blame component, pretending we'd be fine if only "X wasn't this stupid". I read this as a psychological coping mechanism designed to sidestep the existential question - can we ever truly achieve the required level of security guarantees to replace TradFi?

Vanilla crypto usage (e.g. BTC, ETH transfers) is generally as secure as the weakest link between cryptographic attacks, key safety, and consensus layer. This is a fairly formal mathematical surface. Then contracts were introduced, each one creating a new, unique attack surface. It wasn't good enough apparently. We decided the contracts should be arbitrarily changed at runtime, so now every company sporadically calls people together to sign bytes they don't understand, but the Safe UI says are safe. But we were too greedy to call it a day, so we devised protocols that secure billions of dollars using hot wallets (AKA bridges). Automated infrastructure operating off of RPC calls to route billions of dollars.

What we've ended up with in the de-facto state of DeFi is a system of software and network layer trust assumptions similar or inferior to TradFi, without benefiting from its defensive mechanisms - transaction voiding, regulatory compliance, legal accountability, money trail etc. In other words, the on-chain aspects are a distraction for a traditional, completely trusted setup without all its upside.

We need the collective narrative to finally accept the fact that threat actors have an arsenal of 0days which they weaponize to get access to highly privileged systems. With AI assisted/automated exploitation patch gapping 1days has also become a central threat. The fact we don't see 9-fig exploits every day (although it feels so lately) is not because our security is airtight, but due to operational capacity and prioritizing highest EV ops.

The security endgame questions every corp needs to answer are: How many unique exploits or supply chain attacks are needed to compromise critical assets? How do we implement security barriers that cannot be bypassed by device compromise? For example, the security guarantees of a 3/7 MS of top hardware wallets is drastically different from compromising an RPC endpoint. On-chain time locks and volume thresholds add another friction point to bypass and drastically reduce worst case scenarios unless another bypass is chained.

Would a 2/2 DVN setup have made a big difference? I don't know. LZ likely has much greater resources and depth to secure a validator than smaller orgs, and evidently was still compromised. What's almost certain though is that when, not if, 2/2s will get compromised, we'll just see the industry continue moving the goal posts, to cope with security nihilism which has only one (taboo) conclusion: TradFi is far superior than the current mess we call DeFi.

Until we come back to effectively paper-wallet security, we'll continue to witness ever more sophisticated software exploits tarnishing whatever is left of user trust. And if we can't reach that level of confidence while providing a competitive service to TradFi, then I rather we go to zero on our own terms, rather than sent to zero by DPRK hackers.