felix ringe

We have discovered a substantial number of vulnerabilities in the German legal system using LLMs.

These may enable access to confidential documents, travel without an otherwise required visa, enforceable leave for soldiers at NATO’s JSEC, and more.

Similar to computer systems, the written law can contain vulnerabilities that invite exploitation. In our blog post, we share a selection of vulnerabilities, and present potential exploits for each. To our knowledge, none of these have been previously recorded. We share only those vulnerabilities whose exploitation we believe would not cause serious harm.

blog post: felixringe.com/legal-exploits/

Thank you all for the overwhelming response! I'm glad so many of you took the time to share your thoughts. It's getting very late here on the West Coast, so I'll sign off for tonight, but I'll be back tomorrow!
x.com/felixringe/sta…

I believe it will become standard practice to have AI agents reproduce papers before publication.

My ICML AI4Law workshop paper is now on arXiv with such a reproduction: given only the input data and a description of the methodology, I had agents reproduce both the paper’s reported scores and its benchmark datasets for less than seven (!) cents.

I have laid out the broader case for this approach in a blog post (link below). It also includes a guide to creating such reproductions and discusses some failure modes observed in the agent trajectories (available on @harborframework’s Harbor Hub).

I first came across this idea in @ahall_research's “The 100x Research Institution,” in which he briefly proposes it as one of many ways in which AI might change research and its institutions, and somehow it has stayed with me ever since.

Lastly, thank you to the organisers and everyone who stopped by to discuss my poster at @stanfordHAI’s Empirical Methods in the Age of AI conference this weekend!