Twitxy Saved

Tapa Ghosh

Verified account This account is verified. Learn more

@semiDL

Founder & CEO @ Volantis - we’re solving the memory bottleneck by using optics to connect huge amounts of fast memory to chips.
SF Bay Area, CA Volantissemi.ai Joined August 2013
491 Following 3.8K Followers 12K Posts

Latest posts 5

Pinned post

Tapa Ghosh Verified account @semiDL
Excited to announce Volantis's $88M Series A.

We are solving Al's memory bottleneck by using optics, enabling chips with huge amounts of fast & cheap memory.

By boosting both the memory bandwidth and capacity per chip by orders of magnitude, we enable ultra-fast inference (up to 10,000 tps/user) for large models (>10T) - with low $/tok to boot.

Initially, this will enable insanely fast agents - think coding agents that finish in minutes or even seconds instead of hours.

More excitingly, optics is a fundamentally scalable way to increase memory systems. Not 2X/year, but by orders of magnitude across new generations. This will enable a structurally new Al industry, including restarting scaling laws, holding entire repos in context windows & more.

Our team has pioneered many core semiconductor technologies: the 1st CoWoS product, early HBM, the 1st silicon photonics CPO systems, the 1st high volume tunable VCSELs, the 1st processors to directly communicate using light & more.

We’ve already sent data >10× farther than equally tiny electrical wires inside a chip package. Our next iteration is already taped out and targets world-record bandwidth density over relevant distances, read more: volantissemi.ai/news-insights/…
212 replies 182 reposts 1.3K likes 1M views View on X
Tapa Ghosh Verified account @semiDL
Indeed- disaggregated inference and heterogenous throughout/low-latency inference are great if you perfectly know your workload demand mix 1-3 years from now

But for most purchasers that’s a very hard prediction to make
Vincent Zhong Verified account @vincentzed_cuda 16h
A chip that’s 3× faster but idle half the day isn’t 3× cheaper:

A prefill or decode device only pays off when your traffic matches the ratio you bought, and that ratio keeps changing. Your GPUs can switch roles when that happens, but specialized silicon just sits idle while you keep paying for it. Software keeps moving the bar too, since megakernels alone have pushed low-concurrency decode on ordinary GPUs up by 40–75% this year, and every one of those gains is another knock on specialized chips.

Consider: “how fast?” but “how fast, times how often it’s actually busy, after moving the KV cache over the network?” If your GPUs run at 70% and their chip averages 40% across a real year of traffic, it needs to be ~1.75× better just to tie. For a single model with very stable traffic (like a premium fast tier), that can work out, but in many case, general-purpose computing can be more useful.
View quoted post on X
2 replies 1 reposts 32 likes 2.6K views View on X
Tapa Ghosh Verified account @semiDL
People forget that “build a company not a feature” applies to deep tech too

A small innovation, no matter how novel, rarely leads to a gigantic company- it has to be something scalable across multiple generations