Twitxy Saved

Veniamin Veselovsky

Verified account This account is verified. Learn more

@VminVsky

forecasting the future at preseen. prev @princeton
San Francisco theveniamin.com Joined June 2017
483 Following 1K Followers 467 Posts

Latest posts 5

Veniamin Veselovsky Verified account @VminVsky
Super excited to be partnering with @Bill_Gross and Idealab.ai to integrate Preseen into forecasting start up idea success!
Preseen @AskPreseen Oct 19
Preseen is now available on @Idealab.ai to put forecasts in the hands of founders deciding what business to start.

Idealab.ai makes more than 1.2 million business ideas freely searchable, drawing on @Bill_Gross's 50-year entrepreneurial experience.
View quoted post on X
1 replies 0 reposts 12 likes 2.7K views View on X
Veniamin Veselovsky Verified account @VminVsky
Can two superforecasters + machine outperform machine in forecasting?

When the machine first beat a human in chess there was a point in time when man + machine > machine.

At Preseen we’ve built out one of the best AI systems for forecasting. We wanted to see if two superforecasters working with our system could outperform the system itself.

They ended up narrowly beating the system, but what we learned was fairly surprising.

A few takeaways:
- Being a domain expert doesn’t map onto using an AI system well. Guiding superhuman (or at least on-par) AI requires a different skillset. Effectively working with an AI system requires an additional skill: knowing when and how your own expertise can improve its output.
- Evaluating a centaur approach is non-stationary since the human’s behavior with the agent changes as they learn more about the constraints of the agents. If an agent has a few dimensions a human can interact with, the human’s use of those dimensions can change dramatically.
- Human agreement could reinforce a mistake. On the three questions where overrides cost the most, the humans moved toward the community prediction. What looked like a sensible correction supported by other forecasters ended up hurting performance.
6 replies 5 reposts 28 likes 4.4K views View on X
Veniamin Veselovsky Verified account @VminVsky
I’ve been a fan of James’ work since undergrad! Super excited for him to be joining Preseen as our Chief Scientist.
Preseen @AskPreseen Sep 29
James Evans (@profjamesevans) has joined Preseen as Chief Scientist.

He's a @UChicago professor and founding director of Knowledge Lab and the Chicago Center for Computational Social Science (C3S2), studying how people and AI generate knowledge and anticipate discoveries.
View quoted post on X
Veniamin Veselovsky Verified account @VminVsky
A lot of experts quoting probabilities for p(doom) have little to no forecasting background.

We've repeatedly seen that experts have access to information that's relevant for a forecast, but produce uncalibrated forecasts themselves.

For reference, here is Preseen's forecast for "Will AI cause human extinction by December 31, 2100?": preseen.com/q/7Q94Kyncj4XN…
preseen.com Will AI cause human extinction by December 31, 2100? This question asks whether artificial intelligence will cause a catastrophe on or before December 31, 2100 that kills more than 99% of the human population. Despite the use of the word "extinction" i…
D.A. Wallach Verified account @dawallach Sep 28
“One of the main lessons to come out of Tetlock’s work is that the best generalist forecasters are usually better at telling the future than domain-specific experts, even when it comes to those experts’ areas of study. This is because forecasting itself is its own science that requires a unique way of thinking, and a person having a great deal of knowledge about a field is no guarantee that they have good epistemological habits. If you’re going to ask, for example, whether American democracy is falling apart, an expert might start by making a superficial comparison between the modern US and the Weimar Republic. Someone with a forecasting mindset would be more likely to begin with a question like “how often do advanced democracies slide into dictatorships?” The answer is not very often.”

Important reminder from @RichardHanania that domain experts are not the best at predicting what happens, even in their own domains.

open.substack.com/pub/richardhan…
View quoted post on X
0 replies 0 reposts 15 likes 1.3K views View on X
Veniamin Veselovsky Verified account @VminVsky
Forecasting science and technology is hard. When Bell Labs unveiled the transistor in 1948, The New York Times gave it a throwaway item on page 46.

This pattern holds across fields with the best-funded programs and most-cited papers seldom being the ones that matter most, and the advances that do are often ones no one saw coming.

I'm excited Preseen will be partnering with the Knowledge Lab at UChicago to incorporate their data and models into Preseen's core forecasting technology to improve our ability to forecast science and technology.

More on this here: blog.preseen.com/p/predicting-s…
blog.preseen.com Predicting Science and Technology: Preseen Partners with UChicago's Knowledge Lab Technologies that remake the world rarely announce themselves.