Twitxy Saved

Simos Gerasimou

@simosgerasimou

CS Associate Professor @ University of York, UK. 'I have no special talent, I am only passionately curious'
York, United Kingdom www-users.cs.york.ac.uk/simos Joined December 2010
609 Following 429 Followers 1.5K Posts

Latest posts 5

Simos Gerasimou @simosgerasimou
Thanks for your kind words @predict_addict. Indeed, MC-CP, as a hybrid UQ method, brings the best of the two. Despite its simplicity, it yields robust prediction sets/intervals by exploiting the statistical efficiency of MC dropout & CP's distribution-free coverage guarantees๐Ÿš€๐Ÿ”ฅ
Valeriy M., PhD, MBA, CQF Verified account @predict_addict Aug 21, 2023
Conformal Prediction + Monte Carlo is absolute ๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ. Finally researchers applied conformal prediction to Monte Carlo to revive popular method to be fit for the XXIst century and it is shaping to be an absolute ๐Ÿš€๐Ÿš€๐Ÿš€๐Ÿš€๐Ÿš€.

A nice start of the week with release of what looks like very impressive #conformalprediction paper by a team of researchers (Daniel Bethell, @simosgerasimou and Radu Calinescu).

I have been saying for long time ๐Ÿ•ฐ๏ธ that uncertainty quantification methods based on Monte Carlo dropout are non performant in terms of coverage, so it is great to see conformal prediction applied to Monte Carlo to correct its issues.

This work is great and has all the hallmarks to become very popular method, remember Conformalized Quantile Regression invented by @yaniv_romano that also combined classical method such as Quantile Regression with CP resulting in absolute blockbuster of a model.

The new model Monte Carlo-Conformal Prediction (MC-CP) looks like an awesome idea ๐Ÿ’ก and delivers improvements both in classification and regression tasks.

๐Ÿ’  ingenious cross breed hybrid idea to combine Monte Carlo with conformal prediction

๐Ÿ’  adaptively modulates traditional MC dropout at run time to save memory and computational resources

๐Ÿ’  unlike the original MC dropout, MC-CP delivers robust prediction sets and intervals

๐Ÿ’  significant improvement on SOTA methods including not only MC dropout but also some of the current conformal methods such as RAPS by @ml_angelopoulos and CQR

๐Ÿ’  Boosts accuracy in the process whilst delivering robust prediction sets and intervals

๐Ÿ’  Delivers even more confident CP predictions

๐Ÿ’  Highest accuracy across classes, whether it is bird ๐Ÿฆ… cat ๐Ÿฑ deer ๐ŸฆŒ dog ๐Ÿถ frog ๐Ÿธ ship โ›ด๏ธ automobile ๐Ÿš— or horse ๐ŸŽ MC-CP outperformed against other methods.

This paper and methods looks very impressive and if the benefits are confirmed in large scale studies is shaping to be another potential blockbuster conformal prediction model.

arxiv.org/pdf/2308.09647โ€ฆ

Great work by the team from @UniOfYork

#conformalprediction #machinelearning
View quoted post on X