I tried to fool three AI image judges with a vivid but blurry image against a dull but sharp one. They all ignored the pretty one. Then GPT-4o invented answers and Claude kept answering B. 60 image pairs, 1,080 questions. Thread:
Muhammad Abdullahi dattijo
Verified account This account is verified. Learn more@DattijoMuhammad
AI developer | Cybersecurity researcher | Technical writer | Exploring the systems shaping AI, security, and digital policy.
617 Following 105 Followers 245 Posts
Latest posts 5
e. None of the three did. But they walked into a different trap. Thread:
I tested four ways to give an AI assistant a memory. The obvious one, pasting the whole history into every prompt, was the worst value: 20 to 31 times the cost of retrieval, and it did not win on accuracy either. Thread:
I made an apartment search agent 25x cheaper and it still found all 101 true matches with zero false positives. Cost for 500 listings x 5 requirements went from $0.20 to $0.008. Sending every pair to the strong model would have cost $1.88. Here is what actually moved the bill.
I ran an experiment on AI slop in training data. The fix I tried made my model worse. A thread.