Twitxy Saved

Nikhil Murthy

Verified account This account is verified. Learn more

@NikhilKMurthy

co-founder of @phonic_co | cs @MIT
San Francisco, CA phonic.ai Joined September 2017
333 Following 354 Followers 139 Posts

Latest posts 5

Pinned post

Nikhil Murthy Verified account @NikhilKMurthy
A few years ago, Moin and I were driving down 280 and talking about building something in voice AI. We were excited by the future with models that could truly understand and naturally speak with us, and it felt obvious to me at the time that these models, like us, needed to take in and output audio. It turns out this is an extremely hard thing to build, but I’m so proud of the Phonic team for taking us a step closer in that direction with Phonic-1.

Few lessons I learned:
1. Intelligence cannot be traded off for low latency or naturalness: the most valuable workloads in voice are intelligence bottlenecked (multi-step tool calling, instruction following and long context understanding)
2. Real calls are the ultimate benchmark. Whenever I sit with a customer and listen to their calls, I always find new failure modes and learnings that help us build at the frontier.
3. Reliability is everything. A voice call going silent or dropping is truly an awful experience, and you can’t just retry a request (like coding agents). We had to think about our infra from the ground up for reliability.

We’re so excited to release Phonic-1. It’s a really great model across intelligence, latency and naturalness, and already making a huge impact with our customers. And so much more in store to come!
Moin Nadeem Verified account @moinnadeem Oct 29
Introducing Phonic-1, the most advanced speech-to-speech model for agentic tasks — from @phonic_co, the speech-to-speech model company.

We’ve had a breakout year and raised a $27.5M Series A led by @firstround with participation from @Lux_Capital, @felicis, and Definition.
View quoted post on X
5 replies 11 reposts 48 likes 4.1K views View on X
Nikhil Murthy Verified account @NikhilKMurthy
Voice AI can be difficult to benchmark, but DIAL from @withdavidai has done a great job with live, multi-turn spoken conversations with real people.

Great to see Phonic-1 performing so well (#1 on naturalness and content!)
David AI Verified account @withdavidai 2d
New on DIAL: Phonic-1 from @phonic_co and Realtime v1 mini from @ElevenLabs.

Overall preference:
🆕 Phonic-1: #2, 4.04/5
🆕 Realtime v1 mini: #12, 3.60/5

We evaluate these models through live, multi-turn conversations with human raters. Updated overall preference rankings below.

Performance can be separately compared across Consumer and Business conversations, with breakdowns across naturalness, audio quality, content and the issues users flag during their calls.
View quoted post on X
2 replies 1 reposts 22 likes 957 views View on X
Nikhil Murthy Verified account @NikhilKMurthy
an underrated concept in duplex speech-to-speech models is time

there's the time dimension of real-time speech input and output (typically represented by tokens that are 20-200ms encoded audio chunks)

then there's the time dimension of reasoning and tool calls

assume 100ms audio chunks for simplicity (moshi uses 80ms, interaction models uses 200ms). this results in 10 tokens/second to be real-time. if reasoning and tool calls operated at the same "clock speed," a 100 token tool call generation would take ten seconds!!

one way around this is fully delegating so that the frontend duplex model can be fully focused on real-time interactions. this is awkward though, as the main frontend model doesn't have context on tool execution and always has to stall when delegating ("hmm, give me a second").

there are many other ways around this, but thinking about time to maintain real-time understanding and generation while also leaving enough time to reason and take action is critical
Nikhil Murthy Verified account @NikhilKMurthy
I’m always blown away by the kinds of voice-native products and use cases startups build. If you’re one of them, join our startup program to get credits and more!
Phonic Verified account @phonic_co 6d
Introducing The Speech-to-Speech Starter Grant.

Last week, we launched Phonic-1, the most advanced speech-to-speech model for agentic tasks. We now want to get Phonic-1 into the hands of startups that are excited to build products on top of the latest models.

We’ll cover 144,000 minutes of Phonic across 12 months and provide direct access to our research and engineering team.

In return, we’d love to learn from you. Developers building on top of Phonic teach us more about where we should take Phonic than any benchmark.

If you’re a startup looking to build on speech-to-speech models, apply at phonic.ai/startups!
View quoted post on X
2 replies 2 reposts 22 likes 625 views View on X