NeuRealityAI

93% of organizations in a McKinsey survey are already over budget.

Spend can nearly quadruple as companies move to enterprise deployment.

Anthropic's IPO filing discloses that Broadcom will lend it up to $42B to finance leased compute capacity.

The more capital tied up in AI infrastructure, the more every idle GPU and inefficiently served token matters.

More compute doesn't mean more token output.
Our VP of BD Gaurav Shah explains why in @RTInsights:

"Communication is no longer adjacent to compute but has become part of the compute path itself."

In scale-out AI clusters, the real question is how much of your compute is actually working.

Faster GPUs make the problem worse. The more compute you add, the more time it spends waiting on data.

Unless your NIC is built for AI.

Read more👇

AI clusters are filling up with chips that don't share a protocol.

On the SemiWiki Semiconductor Insiders Podcast with @DanielNenni, our CEO @MosheTanach laid out what this costs.

Google runs Falcon.
Microsoft's Maia uses its own RoCE-like protocol.
Nvidia spans NVLink, InfiniBand, and RoCE, and has now absorbed Groq's inference silicon.
Meta has their own RoCE (MetaRoCE).

Mix them in one cluster and something has to translate between them. Every hop through a separate bridge costs latency.

Our NR2 AI-SuperNIC runs one programmable transport engine.

Protocols load in software. RoCE and Ultra Ethernet today, and the engine is programmable enough to take others.

Conversion happens in the NIC instead of a separate hop.

Full conversation linked 👇

Orr Mazor, one of our ML engineers, got NVIDIA and AMD to serve the same inference request.

Prefill ran on an NVIDIA H100.
Decode ran on an AMD MI300X.

Prefill is compute-bound.
Decode is bound by memory bandwidth.

Splitting across vendors also means capacity can come from more than one supply channel, which matters when specific GPUs are scarce.

Read the full blogpost 👇

This week NeuReality hit the beach 🏖️

We take our mission seriously, but also know when to step outside and spend quality time together.

Good food and games, ice baths, professional massages, and plenty of laughs.

A great day with the team, and a great way to head into the new year.

Wishing everyone a Shana Tova U’Metuka, and a happy, healthy and sweet new year with family and friends! 🍎🍯