Ahh ok. Im happy to try to help you pressure the frontier labs to play ball.
Some of them claim to be good enough for military targeting and stuff (even though awful mistakes seem to have happened), why not do a proper real test?
Some of them claim to be good enough for military targeting and stuff (even though awful mistakes seem to have happened), why not do a proper real test?
@TomDwan Great point, Tom. Running frontier LLMs at scale is expensive. That's why we use AIVAT, a variance reduction technique that achieves the same statistical significance in 10x fewer hands, so 5K is equivalent to ~50K raw hands.View quoted post on X