The Main Thread
Every week I take one topic in distributed systems and AI engineering and work it out from first principles. Here is the full archive, grouped by topic.
The Main Thread
I write a weekly deep dive like this one on distributed systems and AI engineering. One issue every week, straight to your inbox.
Distributed Systems Patterns
The Saga Pattern: Distributed Transactions Without 2PC
Jun 2026Distributed transactions without two-phase commit.
Event Sourcing: When, Why, and How (and When Not To)
Jun 2026When an event log beats a mutable state store, and when it doesn't.
Why Your Load Balancer is Solving a Market Design Problem
Jun 2026Load balancing through an economics lens.
Distributed Locks: Patterns, Pitfalls, and the Redlock Controversy
May 2026Why TTL-based locks are unsafe and what fencing tokens fix.
Backpressure: When The System Can't Keep Up
May 2026What to do when work arrives faster than it leaves.
Database Replication Topologies
Apr 2026Leaders, followers, and the trade-offs between topologies.
Circuit Breakers: Building Systems That Survive Downstream Failures
Apr 2026Failing fast when a dependency is down.
Consensus Algorithms: Raft Explained From First Principles
Apr 2026How a cluster agrees on anything at all.
CAP Theorem: What It Actually Means (Not What You Think)
Mar 2026What CAP actually constrains, minus the folklore.
Idempotency: Pattern That Makes Distributed Systems Actually Work
Mar 2026The property that makes retries safe.
Timeouts, Retries, and Deadlines
Mar 2026The primitives that keep requests from hanging forever.
Don't Break Production With a Retry Loop
Feb 2026How well-meant retries amplify outages.
Production LLM Engineering
Observability for LLM Applications: Seeing Inside the Black Box
Jun 2026Tracing and evaluating nondeterministic systems.
Latency Optimization for LLM Applications
Jun 2026Making a 10-second response feel like one.
The True Cost of LLM Applications
May 2026The bill beyond per-token pricing.
Building Reliable AI Agents
May 2026Engineering agents that survive contact with production.
Chunking Strategies for RAG: The Definitive Practical Guide
Apr 2026How you split documents decides what retrieval can find.
Structured Outputs: JSON, Tools, and Function Calling That Actually Work
Mar 2026Getting machine-readable answers reliably.
Context Window Management: The Hidden Engineering Problem
Mar 2026Fitting the right information into a bounded window.
The $0.002 That Decides If Your AI App Makes Money
Feb 2026The unit economics of an LLM call.
Why Non-English Speakers Pay 2x More For LLMs
Jan 2026Tokenization economics across languages.
What Happens in the 200ms After You Hit Enter on Your LLM?
Jan 2026The inference pipeline, end to end.
Embeddings & Vectors
5 Questions to Ask Before Adding a Vector DB
Feb 2026Whether you need one at all.
From 50,000 Dimensions to 384: Compression That Powers AI
Feb 2026How dense vectors compress meaning.
The Geometry of Meaning
Jan 2026What distance and direction mean in embedding space.
The Hidden Order in How We Use Words
Jan 2026The distributional structure of language.
The Algorithmic Evolution of One Powerful Idea
Dec 2025From word counts to learned vectors.
Why Embeddings Confused Me at First
Dec 2025Building intuition for vectors as meaning.
Tokenization
Tokenization: The First Bridge from Language to Thought
Dec 2025Tokenization as representation design, not preprocessing.
Tokenization: Fairness Starts at the Token Level
Dec 2025Why the same sentence costs more tokens in some languages.
Tokenization: How Machines Learn Language Fragments
Dec 2025How subword vocabularies get built.
Tokenization: Before Words Become Numbers
Nov 2025What tokenization is and why models need it.
Caching
Caching: Mastery and Advanced Ops
Nov 2025Operating caches in production.
Caching: Mitigating Inconsistencies
Nov 2025Keeping cache and database in agreement.
Caching: Memory Management, Eviction, and Common Pitfalls
Nov 2025Eviction policies and where they fail.
Caches Lie: Consistency Isn't Free
Oct 2025Why every cache serves stale data and what it costs.
B-Trees & Storage Engines
Lessons Beyond B-Trees: Embracing Systems Thinking
Oct 2025What the series teaches about systems in general.
When 1,000 Threads Hit the Same B-Tree
Oct 2025What concurrency does inside the tree.
Trade-offs Inside B-Trees: Tuning for Real Hardware
Oct 2025Node sizes and fill factors on real disks.
B-Trees Aren't Trees — They're Bandwidth Optimizers
Sep 2025B-Trees exist because disk I/O, not CPU, is the bottleneck.