Twitxy Saved

Cuong Le

Verified account This account is verified. Learn more

@cuongleqq

VictoriaLogs core maintainer @VictoriaMetrics | ex-Head of Engineering @FeralFile | Writing about high-performance Rust, Go & database internals
Joined September 2017
141 Following 264 Followers 127 Posts

Latest posts 5

Cuong Le Verified account @cuongleqq
I spent 3 months benchmarking and digging into the internals of Loki, Elasticsearch, OpenSearch, ClickStack, SigNoz and VictoriaLogs, and condensed it into a 20-minute read.

TL;DR: I realized who ingests fastest is only one side of the story. Every fast number had a price.

I abused each of them with a flood of structured, Kubernetes-style logs on identical machines, until they choked.

1. Loki ingested the most: 139k lines/s.

It indexes only stream labels and keeps lines as plain text, pushing the cost to query time.

Any search or aggregation on a field, the kind every dashboard runs, is expensive and can take minutes: Loki has to decompress and scan every line in matching streams.

2. Elasticsearch finds a single word almost instantly, because it indexes every word of every line.

The price: all that indexing is CPU-heavy, so it topped out at 48k lines/s. And it needed almost 5x the disk of VictoriaLogs.

3. SigNoz and ClickStack store logs in ClickHouse, a columnar database like VictoriaLogs. I expected them to be close. They only reached 10k and 46k lines/s.

The bottleneck wasn't ClickHouse. It was the OpenTelemetry collector in front of it, which used more CPU than ClickHouse itself.

4. VictoriaLogs (I work on it) ingested 120k lines/s with the least memory and disk. It stores each field in its own column and reads only the columns a query needs, so every test query returned within seconds.

The catch: it shines on structured logs, where columns help most. On plain-text logs (from older apps or network devices), it still works, but loses most of its edge.
Aliaksandr Valialkin Verified account @valyala Oct 29
@cuongleqq published data ingestion benchmark results for open-source databases for logs: VictoriaLogs, Loki, Elasticsearch, OpenSearch, SigNoz and ClickStack (the last two are based on ClickHouse).

The benchmark measures CPU usage, memory usage, disk space usage and disk IO usage for different ingestion rates of Kubernetes-like logs. It also measures performance for production-like queries.

Hint: VictoriaLogs isn't the fastest one for data ingestion.

victoriametrics.com/blog/structure…
View quoted post on X
1 replies 1 reposts 23 likes 971 views View on X
Cuong Le Verified account @cuongleqq
Cool video. Postgres handles large scans differently -- and I find this one equally clever.

Its shared_buffers cache uses clock-sweep instead of an LRU list. Each buffer slot has a usage_count. When Postgres needs a victim, it sweeps through slots until it
finds one:
• pinned: skip
• usage_count > 0: decrement and skip
• usage_count == 0 and unpinned: victim found

For large scans (table blocks > 1/4 of shared_buffers slots), Postgres allocates a small ring buffer -- just 256KB, roughly 32 slots for 8KB pages. The scan fills
victims into this ring, then cycles back to reuse them. If a buffer in the ring has become hot (pinned or usage_count > 1), clock-sweep finds a replacement from the main pool instead.

Net effect: a full table scan mostly churns through a tiny ring, typically around 256KB, instead of blowing out the whole shared_buffers cache. Simple idea, surprisingly effective.
Arpit Bhayani Verified account @arpit_bhayani May 5
Table pages are cached, and this is how databases get a performance boost. But, a single full table scan can destroy such cache because, but here's how MySQL cleverly prevents this...

When a full table scan happens, it can flush out all valuable cached pages and replace them with pages that are often never needed again. Given that every single page that is accessed is put into the cache. This 'cache pollution' forces subsequent queries to hit slow disk storage.

MySQL solves this with the particularly simple yet clever midpoint insertion strategy, and I have explained this approach in my video. Give it a watch.

Hope this helps.
View quoted post on X
Cuong Le Verified account @cuongleqq
Something is seriously wrong with how people contribute to open source these days.

People submit AI-generated code they clearly don't understand. When reviewers comment, they feed it back to the AI and submit again. The loop never ends.

Low-effort PR requires extremely high effort to review. That math is never fair.

You think this is how you build a reputation in open source? No, no, and no. You're just burning out maintainers.

---

Use AI to understand the codebase instead. Ask it to explain things. Build your own judgment. It's ok to use AI as a tool for coding, but make sure you filter out any AI slop before submitting.

You gain the skills. You gain the reputation. The project actually benefits.

Is it that hard?
3 replies 2 reposts 19 likes 1.4K views View on X
Cuong Le Verified account @cuongleqq
Contributing to Rust has been a dream of mine for a long time. Last December, I finally did it. A small but real contribution to Rust Clippy. Here's what I learned that no tutorial will teach you.

1. "It works" ≠ "it belongs here"

My code had zero logic errors from the first submission. Tests passed. The lint fired correctly.

It still took multiple review rounds before it was accepted.

Each round, the reviewer flagged something I hadn't considered. A different API, a different factoring, a different order of checks. None were correctness issues. All were belonging issues.

Mature codebases have a shape. A vocabulary. Conventions no doc can fully capture. And when that codebase operates on AST and HIR, the learning curve is steeper than most.

Getting your code to work is step one. Getting it to fit is step two. Only the first is obvious to newcomers.

2. Performance is a first-class design constraint

I spent years building web and backend systems where readability almost always beats micro-optimization.

Clippy changed that.

The reviewer pushed back on how I structured my checks. His version was slightly harder to read, but it was faster. I had optimized for clarity. He optimized for performance. Both were valid, but only one was right for Clippy. The reasoning: this code runs on every file, every build, for every Rust developer in the world. Nanoseconds x millions of daily builds = something real.

Readability-first is a fine default. But in performance-critical code like compilers, runtimes, and toolchains, scale flips the equation. Know which world you're writing for.

3. Don't let ego get in the way of the reviewer

I was surprised by how many specific changes were requested. It felt strict at first.

Then I paused and thought about it. Every request was reasonable.

The reviewer has lived with this codebase. He understands every design decision and genuinely cares about keeping it in good shape. His knowledge was the most valuable resource I had access to, and I almost let pride get in the way of using it.

In a review from someone who knows the codebase far better than you: every comment is a gift. Receive it that way.

The flip side: if you're ever reviewing a first contribution, that same dynamic works in reverse. Your patience, or lack of it, is the whole experience for them.

4. A good test suite makes new contributors brave

My biggest fear as a first-timer: breaking something silently.

Clippy's golden-file tests killed that fear. The exact expected output is checked in alongside the code. Shift a diagnostic message even slightly, the test fails and shows you exactly what changed. Clippy even runs its own lints on itself.

Knowing the safety net was that solid made me willing to experiment instead of playing it safe.

A comprehensive test suite isn't just quality assurance. For anyone contributing to an open source project, it's a confidence multiplier. The difference between experimenting freely and playing it safe.

---

That was my first time contributing to a real, large-scale open source project. Small contribution. Big lessons. More to come as I go deeper.
0 replies 2 reposts 22 likes 1.3K views View on X
Cuong Le Verified account @cuongleqq
Rust Beats C++ by a mile

security.googleblog.com/2025/11/rust-i…

The numbers are crazy IMO.
• Rust: ~0.2 memory bugs per million lines, C/C++: ~1000 per million
• Rust code reviews move ~25% faster
• ~20% fewer review revisions
• Rust changes get reverted far less often, roughly 4x better

Google is pushing Rust into more of Android: drivers, firmware, the whole stack
blog.google Rust in Android: move fast and fix things