StarTree

𝗪𝗵𝗮𝘁’𝘀 𝗵𝗮𝗽𝗽𝗲𝗻𝗶𝗻𝗴 𝗻𝗼𝘄 𝗮𝘁 @wearemiq?
MiQ is reinventing how programmatic advertising campaigns get built—
using AI to unify audience discovery, segment selection, and campaign activation across fragmented DSP ecosystems.

And it’s 𝗽𝗼𝘄𝗲𝗿𝗲𝗱 𝗯𝘆 𝗔𝗽𝗮𝗰𝗵𝗲 𝗣𝗶𝗻𝗼𝘁.

Because in modern advertising, the bottleneck isn’t access to data.
It’s the ability to search, compare, and activate audiences in real time across massive, fragmented systems.

𝗧𝗵𝗲 𝗰𝗵𝗮𝗹𝗹𝗲𝗻𝗴𝗲
MiQ manages more than 100,000 audience segments across multiple providers and DSPs.

But before Audiences:
  • Traders manually stitched together segment data across disconnected systems
  • Sales teams struggled to turn insights into activatable campaigns
  • Slow query performance created friction in high-speed workflows

Traditional architectures couldn’t handle the concurrency and responsiveness required for real-time audience exploration

𝗧𝗵𝗲 𝗶𝗻𝘀𝗶𝗴𝗵𝘁
MiQ built a unified, AI-driven Audiences platform powered by Apache Pinot.
  • Streaming and indexed audience data now enables real-time search across massive segment inventories
  • Free-text audience discovery powered by vector indexing and LLM-generated metadata
  • Instant comparison of reach, CPM, and availability across DSPs
  • Direct activation workflows from discovery to execution

This transforms campaign building from a fragmented manual process into an intelligent, interactive system.

𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁𝘀
  • Segment listing latency reduced from 8–10 seconds to ~2 seconds
  • Complex metric calculations accelerated by 40–80%
  • Query caching eliminated entirely
  • Multiple users can simultaneously explore audiences without performance degradation
  • AI-powered discovery improves campaign planning and audience selection

𝗧𝗵𝗲 𝗯𝗶𝗴𝗴𝗲𝗿 𝘀𝗵𝗶𝗳𝘁
Programmatic advertising is moving from static workflows to intelligent systems that reason across fragmented data in real time.
Because AI isn’t just changing ad targeting.

It’s changing how campaigns themselves are constructed.

𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗰𝗮𝘀𝗲 𝘀𝘁𝘂𝗱𝘆 𝗵𝗲𝗿𝗲 → stree.ai/4tvKdcW

𝗪𝗵𝗮𝘁’𝘀 𝗵𝗮𝗽𝗽𝗲𝗻𝗶𝗻𝗴 𝗻𝗼𝘄 𝗮𝘁 @Webex?
Webex is building real-time observability for one of the world’s largest collaboration platforms—where engineers can detect audio degradation, latency spikes, and platform anomalies as they happen.

𝗔𝗻𝗱 𝗶𝘁’𝘀 𝗽𝗼𝘄𝗲𝗿𝗲𝗱 𝗯𝘆 𝗔𝗽𝗮𝗰𝗵𝗲 𝗣𝗶𝗻𝗼𝘁.

Because at Webex scale, observability can’t rely on static metrics or delayed rollups.

You need runtime analytics across billions of events, under concurrency, with fresh data arriving continuously.

𝗧𝗵𝗲 𝗰𝗵𝗮𝗹𝗹𝗲𝗻𝗴𝗲

As remote work exploded, Webex had to support:
  • 100+ TB of telemetry data per day
  • Over 300,000 messages per second at peak
  • More than a billion events daily
  • Hundreds of dimensions across audio quality, regions, clients, and user behavior

The existing #Elasticsearch-based architecture struggled under the load:
  • Slow queries
  • Timeouts under concurrency
  • Heavy infrastructure costs
  • Rollups that limited visibility into emerging problems

And in #observability, pre-aggregated data misses the very anomalies you’re trying to detect.

𝗧𝗵𝗲 𝗶𝗻𝘀𝗶𝗴𝗵𝘁

Webex rebuilt its observability platform around Apache Pinot.

Streaming telemetry now powers:
  • Real-time runtime aggregations across raw event streams
  • Sub-second exploration of audio/video quality metrics
  • High-concurrency analytical queries across hundreds of dimensions
  • Live dashboards and alerting integrated with #Grafana and #Kibana

This transforms observability from retrospective reporting into an interactive operational system.

𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁

  • 5× to 150× faster p99 query latency compared to Elasticsearch
  • Sub-second query performance in most workloads
  • Elasticsearch timed out in 67% of benchmark cases where Pinot succeeded
  • Cluster footprint reduced by 500+ nodes
  • Data storage reduced from 800TB to 121TB of unique data

𝗧𝗵𝗲 𝗯𝗶𝗴𝗴𝗲𝗿 𝘀𝗵𝗶𝗳𝘁

Modern observability systems can’t depend on pre-computed summaries anymore.

Because when infrastructure behavior changes in seconds, the analytics layer must detect and explain anomalies as they emerge—not after the incident is over.

𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗰𝗮𝘀𝗲 𝘀𝘁𝘂𝗱𝘆 𝗵𝗲𝗿𝗲 → stree.ai/4uDn9KR

𝗪𝗵𝗮𝘁’𝘀 𝗵𝗮𝗽𝗽𝗲𝗻𝗶𝗻𝗴 𝗻𝗼𝘄 𝗮𝘁 @stripe?
Stripe is building real-time financial infrastructure that doesn’t just process payments—it explains what’s happening across billions of transactions as events unfold.

And it’s 𝗽𝗼𝘄𝗲𝗿𝗲𝗱 𝗯𝘆 𝗔𝗽𝗮𝗰𝗵𝗲 𝗣𝗶𝗻𝗼𝘁.

Because at Stripe’s scale, analytics can’t be an afterthought.
Customer dashboards, fraud monitoring, billing analytics, and operational alerts all depend on fresh data under massive concurrency.

𝗧𝗵𝗲 𝗰𝗵𝗮𝗹𝗹𝗲𝗻𝗴𝗲
Stripe processes more than 250 million API requests per day, peaking at 13,000 requests per second.

That created a new requirement:
  • Real-time dashboards for merchants and developers
  • Instant visibility into payment processor failures
  • Live financial reporting and risk monitoring
  • Sub-second analytics across petabytes of transaction data

Traditional architectures struggled to balance freshness, latency, and scale simultaneously.

𝗧𝗵𝗲 𝗶𝗻𝘀𝗶𝗴𝗵𝘁
Stripe standardized on Apache Pinot as its real-time analytical layer.

Streaming data flows through Kafka and Flink into Pinot, where:
Customer-facing dashboards update in near real time.

Billing and API analytics stay interactive under heavy load.

Internal teams monitor fraud, risk, and payment infrastructure live.

Queries execute across massive transaction volumes with low tail latency.

This transforms operational payment data into a system that can be interrogated continuously—not just reported on after the fact.

𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁
  • 10,000+ queries per second
  • 70ms p99 query latency
  • 30-second p99 ingestion lag
  • 99.99% availability

Over 1 petabyte of data managed across production Pinot clusters

During Black Friday–Cyber Monday alone, Stripe used Pinot to track 300M+ transactions totaling more than $18.6B.

𝗧𝗵𝗲 𝗯𝗶𝗴𝗴𝗲𝗿 𝘀𝗵𝗶𝗳𝘁
Modern financial platforms aren’t just transaction systems anymore.
They’re real-time analytical systems operating under extreme concurrency.

Because when money moves globally in milliseconds, the analytics layer has to move just as fast.

𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗰𝗮𝘀𝗲 𝘀𝘁𝘂𝗱𝘆 𝗵𝗲𝗿𝗲 → stree.ai/4uB0UVs

𝗦𝗲𝗲 𝘆𝗼𝘂 𝗶𝗻 𝗧𝗼𝗿𝗼𝗻𝘁𝗼 𝗳𝗼𝗿 @confluentinc's 𝗔𝗜 𝗗𝗮𝘆 𝗮𝘁 𝗗𝗮𝘁𝗮 𝗦𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗪𝗼𝗿𝗹𝗱 𝗧𝗼𝘂𝗿.

On Tuesday, May 26, meet the StarTree team at our booth or join Chad Meley for Real-Time Intelligence of Tokens at Scale.

The session looks at what it takes to make streaming, high-cardinality token data queryable in seconds, so teams can inspect live workloads, understand model and user behavior, and troubleshoot AI systems with fresh data and low-latency queries.

See you in Toronto.

𝗦𝗲𝗲 𝗲𝘃𝗲𝗻𝘁 𝗱𝗲𝘁𝗮𝗶𝗹𝘀 → stree.ai/4djPGxE

#DSWT26 #DataStreamingWorldTour #ApachePinot #RealTimeAnalytics #DataEngineering #ApacheKafka

𝗪𝗵𝗮𝘁’𝘀 𝗵𝗮𝗽𝗽𝗲𝗻𝗶𝗻𝗴 𝗻𝗼𝘄 𝗮𝘁 @togethercompute?
Together AI is building observability for the AI era—
where infrastructure teams can understand not just how many tokens were consumed, but why workloads behave the way they do in real time.

And it’s 𝗽𝗼𝘄𝗲𝗿𝗲𝗱 𝗯𝘆 𝗔𝗽𝗮𝗰𝗵𝗲 𝗣𝗶𝗻𝗼𝘁.

Because in LLM infrastructure, dashboards aren’t enough.
𝗬𝗼𝘂 𝗻𝗲𝗲𝗱 𝗵𝗶𝗴𝗵-𝗰𝗮𝗿𝗱𝗶𝗻𝗮𝗹𝗶𝘁𝘆 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀 𝗮𝗰𝗿𝗼𝘀𝘀 𝗯𝗶𝗹𝗹𝗶𝗼𝗻𝘀 𝗼𝗳 𝗲𝘃𝗲𝗻𝘁𝘀, 𝘂𝗻𝗱𝗲𝗿 𝗰𝗼𝗻𝗰𝘂𝗿𝗿𝗲𝗻𝗰𝘆, 𝘄𝗶𝘁𝗵 𝗳𝗿𝗲𝘀𝗵𝗻𝗲𝘀𝘀 𝗺𝗲𝗮𝘀𝘂𝗿𝗲𝗱 𝗶𝗻 𝘀𝗲𝗰𝗼𝗻𝗱𝘀—𝗻𝗼𝘁 𝗵𝗼𝘂𝗿𝘀.

𝗧𝗵𝗲 𝗰𝗵𝗮𝗹𝗹𝗲𝗻𝗴𝗲
As token volumes surged into the billions per hour, Together AI hit a new problem:

Traditional analytics systems weren’t designed for real-time LLM observability.

Customers wanted live usage dashboards by prompt, model, and API key.

Engineers needed to debug latency spikes and optimize GPU allocation in real time.

Finance teams required precise token-level attribution for billing and cost management.

𝗕𝘂𝘁 𝗺𝗼𝘀𝘁 𝗼𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝘀𝘁𝗮𝗰𝗸𝘀 𝗳𝗼𝗿𝗰𝗲 𝗮 𝘁𝗿𝗮𝗱𝗲𝗼𝗳𝗳:
𝙀𝙞𝙩𝙝𝙚𝙧 𝙝𝙞𝙜𝙝 𝙛𝙧𝙚𝙨𝙝𝙣𝙚𝙨𝙨 𝙬𝙞𝙩𝙝 𝙡𝙤𝙬 𝙜𝙧𝙖𝙣𝙪𝙡𝙖𝙧𝙞𝙩𝙮—𝙤𝙧 𝙙𝙚𝙚𝙥 𝙖𝙣𝙖𝙡𝙮𝙨𝙞𝙨 𝙬𝙞𝙩𝙝 𝙨𝙡𝙤𝙬 𝙗𝙖𝙩𝙘𝙝 𝙥𝙞𝙥𝙚𝙡𝙞𝙣𝙚𝙨.

𝗧𝗵𝗲 𝗶𝗻𝘀𝗶𝗴𝗵𝘁
Together AI centralized streaming LLM telemetry into a real-time analytical layer using StarTree, powered by Apache Pinot.

Streaming data flows into Pinot, where billions of token events become queryable in seconds.

Usage can be sliced by model, user, API key, region, and prompt.
Queries reconstruct infrastructure behavior as events unfold.

Text indexing enables prompt-level debugging and anomaly detection.

This transforms LLM telemetry from static batch reporting into an operational system for AI infrastructure.

𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁
  • Sub-second query latency across billions of token events
  • 10-second freshness windows for near real-time visibility
  • High-cardinality analytics at production scale
  • 50%+ storage cost reduction with tiered storage optimization
  • Latency improvements from 10 seconds to 7 milliseconds using Star-Tree indexing

𝗧𝗵𝗲 𝗯𝗶𝗴𝗴𝗲𝗿 𝘀𝗵𝗶𝗳𝘁
LLM observability is becoming part of the product experience itself.
Because when AI infrastructure becomes customer-facing, telemetry can’t arrive tomorrow.

It has to explain what’s 𝙝𝙖𝙥𝙥𝙚𝙣𝙞𝙣𝙜 𝙣𝙤𝙬.

𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗳𝘂𝗹𝗹 𝗰𝗮𝘀𝗲 𝘀𝘁𝘂𝗱𝘆 𝗵𝗲𝗿𝗲 → stree.ai/4draymK

#LLMobservability #RealTimeAnalytics #DataEngineering #ApachePinot