Twitxy Saved

Mike Henry

Verified account Parasail

@mikehenry8484

Founder and CEO of Parasail.io (@parasail_io). Previously Founder/CEO of Mythic and CPO at Groq.
San Francisco Bay Area Joined August 2023
211 Following 386 Followers 205 Posts

Latest posts 5

Mike Henry Verified account @mikehenry8484
Positron's Titan is joining the Parasail fleet. 5MW. Trillions of tokens a day of cost-optimized compute by late 2027. @parasail_io @positron_ai

@parasail_io is the first multi-silicon inference cloud that routes your prompts to the best chip for the job, serving trillions of tokens a day to hundreds of customers. Positron fills a vital part of compute spectrum on the cost optimized side: 4X better cost-per token than next gen NVIDIA/AMD systems which is perfect for large-scale agent swarms, background agents, and physical AI.

Positron has done incredible work building a chip and system that orchestrates across DRAM and SRAM allowing for massively scalable memory pools with commodity memory giving the best of both worlds for memory capacity and bandwidth.

We've been excited to partner with them in the early days through the launch of their Atlas chip and now big capacity plans with Titan.
@positron_ai
1:18
47 replies 18 reposts 116 likes 214K views View on X
Mike Henry Verified account @mikehenry8484
Congrats to @MeghanaMadhyas1 and Ben for the great talk on the many layers of agents we use to drive top tier performance with a small and mighty team.
Parasail Verified account @parasail_io 2d
Our agents tune LLM inference on production GPUs. Here’s what had to be built for that to work:
- GPU leases + a queue so experiments don’t steal customer capacity
- Cloned deployments on mirrored live traffic
- Canaries with auto rollback and a human sign-off

Checkout @MeghanaMadhyas1 and Ben Segal's talk at @QCon later today to learn more about our autonomous LLM performance optimization system.
View quoted post on X
2 replies 1 reposts 10 likes 509 views View on X
Mike Henry Verified account @mikehenry8484
Announcing the first of many GPU clusters that we own and operate ourselves. Given the volatility of the market, a hybrid strategy of owning compute and renting from the utility-scale GPU providers (i.e. not Neoclouds trying to claim software margins) is the way to deliver the best economics and flexible capacity to our customers.
Parasail Verified account @parasail_io Nov 12
Our newest owned cluster is live: NVIDIA B300s in Denver, in the @parasail_io fleet the day after burn-in.

We have a mixed fleet of renting & owned capacity. Rented lets us scale fast and pick up new silicon as it lands. Owned clusters give us a base we control and tune for the largest customer workloads. An inference cloud locked into one way of getting compute gets stuck when the winds change.
View quoted post on X
8 replies 3 reposts 39 likes 2.7K views View on X
Mike Henry Verified account @mikehenry8484
It feels like the 3-4X is achieved where the KV cache is very full and starts missing a lot. Is this just another way of saying it increases effective KV cache space by ~10%?
Weili Xu Verified account @_weilix Nov 9
Introducing FlashState ⚡: an inference system that flexibly manages the prefix cache of hybrid LLMs, designed for agentic workloads.

On GLM 5.3 Flash, FlashState more than doubles vLLM's throughput and provider profit, while cutting latency by 3.7× and user cost by 2.5×.
View quoted post on X
Mike Henry Verified account @mikehenry8484
Solving utilization for heterogenous compute is the defining problem and a lot going on at @parasail_io behind the scenes on this. Stay tuned for some big launches later this year.
Vincent Zhong Verified account @vincentzed_cuda Nov 7
A chip that’s 3× faster but idle half the day isn’t 3× cheaper:

A prefill or decode device only pays off when your traffic matches the ratio you bought, and that ratio keeps changing. Your GPUs can switch roles when that happens, but specialized silicon just sits idle while you keep paying for it. Software keeps moving the bar too, since megakernels alone have pushed low-concurrency decode on ordinary GPUs up by 40–75% this year, and every one of those gains is another knock on specialized chips.

Consider: “how fast?” but “how fast, times how often it’s actually busy, after moving the KV cache over the network?” If your GPUs run at 70% and their chip averages 40% across a real year of traffic, it needs to be ~1.75× better just to tie. For a single model with very stable traffic (like a premium fast tier), that can work out, but in many case, general-purpose computing can be more useful.
View quoted post on X