Bobby Samuels
Verified account This account is verified. Learn more@BobbySamuels
376 Following 767 Followers 199 Posts
Latest posts 5
At @withprotegeai, we’re seeing what many others in healthcare are seeing: There’s a huge clinical readiness gap - the gap between how health AI performs in test settings and how it performs in clinical workflows.
Leading academic institutions and health systems alike all face a similar issue: How do we best measure AI for healthcare applications when AI is already being used by doctors and patients alike?
On some healthcare tasks, AI already performs as well as the best clinicians in the world. On others, the model struggles to recall a simple lab result from a simple report.
Protege Co-Founder and Chief Scientific Officer @engyziedan wrote about this for The AI Collective in their most recent newsletter — in it, Engy explains how she and the rest of the @datalabresearch team approach this challenge from a few different perspective:
#1 - Test Design: Does the test measure the “right” answer?
#2 - Data: Does the data represent real-world scenarios?
#3 - The Referee: Who decides which model is best?
#4 - Gamification: What happens when passing the test becomes the goal?
As the industry moves beyond static test questions and toward realistic, multi-turn scenarios built on real-world healthcare data, we increasingly need an independent evaluator of how AI models are performing.
Check out Engy’s full write-up with The AI Collective here: newsletter.aicollective.com/p/how-are-we-m…
Leading academic institutions and health systems alike all face a similar issue: How do we best measure AI for healthcare applications when AI is already being used by doctors and patients alike?
On some healthcare tasks, AI already performs as well as the best clinicians in the world. On others, the model struggles to recall a simple lab result from a simple report.
Protege Co-Founder and Chief Scientific Officer @engyziedan wrote about this for The AI Collective in their most recent newsletter — in it, Engy explains how she and the rest of the @datalabresearch team approach this challenge from a few different perspective:
#1 - Test Design: Does the test measure the “right” answer?
#2 - Data: Does the data represent real-world scenarios?
#3 - The Referee: Who decides which model is best?
#4 - Gamification: What happens when passing the test becomes the goal?
As the industry moves beyond static test questions and toward realistic, multi-turn scenarios built on real-world healthcare data, we increasingly need an independent evaluator of how AI models are performing.
Check out Engy’s full write-up with The AI Collective here: newsletter.aicollective.com/p/how-are-we-m…
Spirit Airlines made headlines this week after it sold its company data to Google for $10 million. What makes this data valuable is the value of the people’s real output from actual work environments.
What we shouldn’t miss - and what I shared with @JamesJNani of @BLaw in their analysis of the data purchase - is the need for trust and maintaining privacy in what the data represents.
The market for company data is heating up. The core tenets of data privacy, such as de-identification, consumer privacy, and employee protections, all need to be carefully preserved here.
This is something we’re thinking about, especially since many of our backgrounds come from sensitive industries like healthcare where privacy is so important. Here too, the spirit of the law matters just as much as the letter. That’s the standard we’re building toward at @withprotegeai: real-world data, handled with care, and used to make AI effective and people more productive.
Check out the full article here: news.bloomberglaw.com/bankruptcy-law…
Thanks to James and Bloomberg for covering this and speaking with me.
What we shouldn’t miss - and what I shared with @JamesJNani of @BLaw in their analysis of the data purchase - is the need for trust and maintaining privacy in what the data represents.
The market for company data is heating up. The core tenets of data privacy, such as de-identification, consumer privacy, and employee protections, all need to be carefully preserved here.
This is something we’re thinking about, especially since many of our backgrounds come from sensitive industries like healthcare where privacy is so important. Here too, the spirit of the law matters just as much as the letter. That’s the standard we’re building toward at @withprotegeai: real-world data, handled with care, and used to make AI effective and people more productive.
Check out the full article here: news.bloomberglaw.com/bankruptcy-law…
Thanks to James and Bloomberg for covering this and speaking with me.
full article: forbes.com/sites/cortneyh…
www.forbes.com The Future Of AI Training Data Is Human. The Question Is How A new partnership between metaverse startup VLGE and data firm Protege leverages natural human behavioral data from virtual environments to build training sets. @withprotegeai was just featured in @Forbes in a piece called “The Future of AI Training Data Is Human. The Question is How” by @cortneyharding. I wanted to take a minute to highlight how this reflects our core company values: that trust and quality are even more important for this next era of AI development.View quoted post on X
We have a strong stance on what quality data looks like for AI training, evaluation, and application. As the piece notes, “AI is only as good as the metadata that powers it.” We’ve seen how the best data is data that reflects the real world - its quirks, nuances, and human behavior alike.
Every leading AI lab has come to this realization, and that’s how we’ve grown so quickly across multiple domains. At Protege, we value data that is truly representative of ground truth human behavior, whether that’s in video, healthcare, audio, or 3D and world models.
Our partnership with @vlge__ reflects this philosophy. VLGE’s data captures how people actually move, explore, hesitate, and decide while pursuing their own goals inside spatial worlds. As Grant Murphy-Herndon, our GM of Spatial and Physical Intelligence, says in the article: most datasets are "only as good as however you ask people to do it... versus a data set like VLGE where people are kind of going and pursuing their own goals."
That makes the data more informative of real behavior, and inherently less biased. And the data has been collected in a compliant way. This is a key part of how Protege operates: we believe that ethics and data quality point in the same direction.
Ultimately, that's the part I care about most. The next phase of AI will be built on real, lived human experience. Whether that data gets sourced and governed responsibly — this cannot get lost in the shuffle. This is true whether you’re a media rights holder, healthcare system, or spatial data provider.
Thanks to Forbes and Cortney Harding for the thoughtful piece. Worth a read (link in thread)