Latency and cost decide whether open models work in production. That's the work we do every day at Inco.
Today @zhijianliu is on the KV cache panel at #InferCon, 10:55am in Room B. If you're there, go catch it.
Today @zhijianliu is on the KV cache panel at #InferCon, 10:55am in Room B. If you're there, go catch it.
Infer/Con is today 🧡View quoted post on X
10:55am, Room B: our co-founder @zhijianliu is on the "KV Cache as a Distributed Systems Problem" panel at #InferCon, convened by @datadoghq.
At the speaker lunch, he's also hosting a lunch table on speculative decoding and inference for agents. Come join us!