·31 min read
Understanding Embeddings: Part 4 - The Limits of Static Embeddings
Why one vector per word cannot capture meaning—and what this limitation reveals about the nature of language understanding.
EmbeddingsPolysemyContextual Embeddings
Blog
Technical deep dives on distributed systems, databases, performance, and infrastructure. 9 articles and counting.
9 articles
·43 min read
Understanding Embeddings: Part 3 - Global Statistics and GloVe
From local context windows to global co-occurrence matrices—understanding how GloVe unifies prediction and counting methods.
EmbeddingsGloVeMatrix Factorization
·40 min read
Understanding Embeddings: Part 2 - Learning Meaning from Context
Where do embeddings actually come from? Understanding Word2Vec, Skip-Gram, CBOW, and the learning dynamics that create semantic geometry.
EmbeddingsWord2VecCBOW
·34 min read
Understanding Embeddings: Part 1 - Why Embeddings Exist?
Deep dive into embeddings from first principles
EmbeddingsLLMsProduction ML
·21 min read
Demystifying Tokenization: Part 3 - Building & Auditing Tokenizers
Practical guide to building, tuning, and auditing tokenizers: metrics, fairness, production deployment, and real-world lessons
TokenizationLLMsProduction ML
·33 min read
Demystifying Tokenization: Part 2 - Algorithms from BPE to Unigram
Complete implementations of tokenization algorithms: character-level, word-level, BPE, byte-level BPE, WordPiece, and Unigram Language Models
TokenizationLLMsAlgorithms
·20 min read
Demystifying Tokenization: Part 1 - Why Tokenization Matters
Understanding tokenization from first principles: the problem space, design goals, and why it's representation design, not just preprocessing
TokenizationLLMsMachine Learning
·28 min read
Caches Lie: Consistency Isn't Free
Why caching is not just "faster reads"
CachingStorage
·24 min read
B-Trees Aren’t Trees — They’re Bandwidth Optimizers
The biggest bottleneck in databases are not CPU or memory - it's Disk I/O.
DatabasesIndexingB-Trees
Get new articles in your inbox
Technical deep dives on systems, databases, and infrastructure. No spam, unsubscribe anytime.
Join The Main Thread - insights on distributed systems and backend engineering.
Powered by beehiiv
Powered by beehiiv