Pinned post
KORA
Verified account This account is verified. Learn more@meetKORA
The Operating System for Intelligence
2.9K Following 9K Followers 4.4K Posts
Latest posts 5
simplywall.st Qualcomm (QCOM) Deepens Its AI PC Ecosystem With A New Linux Desktop OS QUALCOMM (NasdaqGS:QCOM) is partnering with Kora Systems on a new AI-focused Linux desktop OS for Snapdragon-based PCs. Kora Systems, led by Cyanogen co-founder Kirt McMaster, is tuning the platform... 
"Cortex can schedule AI workloads across the CPU, GPU, and NPU, as well as other systems on the local network, which could be a novel capability if it works beyond the Qualcomm hardware."
"When asked about model formats, McMaster said, “we predominantly support GGUF,” llama.cpp’s model file format, and that Lemma, the inference engine inside Cortex, “can split model layers across processors as needed.” Lemma is built on llama.cpp, whisper.cpp, stable-diffusion.cpp, and ONNX. McMaster says that which runtime is used depends on the model's requirements, and more runtimes will be added down the line."
"McMaster said that with the model, the X2 Elite reaches “25 TPS decode throughput and 550 TPS prefill throughput,” or tokens per second, running “entirely on the Hexagon NPU with 32K context.” A dense model, rather than a mixture-of-experts, would be far slower. Not given are the RAM configuration or the specific X2 Elite chip."
"When asked about model formats, McMaster said, “we predominantly support GGUF,” llama.cpp’s model file format, and that Lemma, the inference engine inside Cortex, “can split model layers across processors as needed.” Lemma is built on llama.cpp, whisper.cpp, stable-diffusion.cpp, and ONNX. McMaster says that which runtime is used depends on the model's requirements, and more runtimes will be added down the line."
"McMaster said that with the model, the X2 Elite reaches “25 TPS decode throughput and 550 TPS prefill throughput,” or tokens per second, running “entirely on the Hexagon NPU with 32K context.” A dense model, rather than a mixture-of-experts, would be far slower. Not given are the RAM configuration or the specific X2 Elite chip."
Kora is a Linux OS that splits AI models across the CPU, GPU, and NPU tomshardware.com/software/linux…View quoted post on X
