Twitxy Saved

jooman ➡️ working on AI inference

Verified account This account is verified. Learn more

@jumanzi_dev

Trustless is what we need in AI epoch / CEO at @fractalyze_io , Ex CTO @a41_allforone, Samsung Research, @DecipherGlobal
seoul Joined December 2021
369 Following 1.1K Followers 154 Posts

Latest posts 5

jooman ➡️ working on AI inference Verified account @jumanzi_dev
Everyone can use the same models. BUT, They don’t have your workload data.

That data tells you where you can reuse computation, which tasks need less intelligence, and how to tune your inference stack.

Why don't you user your own weapon? you can reduce up to 80% of your cost.
Fractalyze ❄️ Verified account @fractalyze_io Oct 12
1/ OpenRouter's cheapest price is not your lowest inference cost.

That price was set for everyone who shares the endpoint. A stack tuned to your own traffic can cost far less.

What that takes, and what we've measured:
View quoted post on X
jooman ➡️ working on AI inference Verified account @jumanzi_dev
You can *own* the current frontier lab's intelligence only after 6m.

opened, commoditized super intelligence, localized + personalized ai is so much close now. inevitable.
Fractalyze ❄️ Verified account @fractalyze_io Sep 30
We made Qwen-Image 2.1 the fastest we’ve seen on a single RTX 5090.
👉 13.5s → 1.9s per image (7.1× faster.)
👉 ~86% less GPU time per image.

How?
→ Kernel rewrites
→ Smarter step caching
→ Faster VAE & text encoder

What model should we break next?
Leave a comment!
View quoted post on X
jooman ➡️ working on AI inference Verified account @jumanzi_dev
higher! we now use our tech into AI inference.
vLLM Verified account @vllm_project Oct 5
Great work from @fractalyze_io optimizing Qwen3-Omni on vLLM-Omni for a single RTX 5090. Their AWQ-4bit, batch-1 text-prompt tests cut time to first audio from 213ms to 23ms vs. stock vLLM-Omni.
We’d love to see these optimizations contributed upstream to vLLM-Omni so more users can benefit!😄
View quoted post on X