New paper introducing a framework to better quantify uncertainty in LLM evaluations (led by @LLuettgau🙌). A beta Python package (developed by @HarryCoppock🚀) is available if you want to try it out.
➡️Get in touch if you have any Qs/feedback!
Paper: arxiv.org/abs/2505.05602
➡️Get in touch if you have any Qs/feedback!
Paper: arxiv.org/abs/2505.05602
Advanced AI systems require complex evaluations to measure abilities, but conventional analysis techniques often fall short.
Introducing HiBayES: a flexible, robust statistical modelling framework that accounts for the nuances & hierarchical structure of advanced evaluations.View quoted post on X
