Writing

Notes on LLM systems, semantic search, embeddings, retrieval, attention, PyTorch, and NLP engineering.

Topics

LLM Architecture Attention PyTorch Decoding Search Embeddings Semantic Search Retrieval NLP Systems Text Processing

Implementation Notes

Concise explanations for specific technical problems, with examples and implementation details.

Two unrelated passages scoring 0.74 cosine similarity, caused by the 0.01 InfoNCE temperature

Why Embedding Models Give High Cosine Similarity Scores for Unrelated Text

Unrelated text scores 0.7+ with BGE, E5 and GTE. The reason is the InfoNCE temperature, not overfitting. A first principles walkthrough with the math.

Embeddings InfoNCE Contrastive Learning Information Retrieval

September 17, 2026 Read →

Illustration of data filtering and preprocessing for LLM training, showing raw text tokens being refined into a high-qua

Technical Note

What I Learned from FineWeb’s 15T Token Recipe

Lessons from HuggingFace's FineWeb 15T-token recipe: base filtering, why more deduplication can hurt, C4 filters, and how FineWeb-Edu picks quality.

Open Source LLM Training Large Language Models NLP

June 28, 2026 Read →

Trie-based constrained decoding restricting an LLM to a fixed label taxonomy

Technical Note

Constrained Decoding: Forcing LLMs to Respect Your Taxonomy

Guarantee an LLM only emits labels from your taxonomy: mask invalid logits to negative infinity and drive the choice with a tokenization-aware trie.

Constrained Decoding Generative AI Decoding Large Language Models

May 05, 2026 Read →

Technical Note

Gemma 3 Explained: How Google Built a Faster, Leaner Multimodal AI

How Gemma 3 got faster and leaner: SigLIP over CLIP, 128K context by interleaving, grouped-query attention, QK-Norm, and quantization-aware training.

LLM Architecture Gemma 3 Multimodal NLP

February 01, 2026 Read →

Attention score matrix with future positions masked to negative infinity

Technical Note

Why Do We Use Negative Infinity for Masking in Attention?

Why transformer attention masks future tokens with negative infinity instead of zero, what softmax does to each, and how to implement it in PyTorch.

PyTorch Transformers Attention Mechanism Deep Learning

December 23, 2025 Read →

Python comparing NFC and NFD forms of the word resume, which are unequal until normalized

Technical Note

A Simple Guide to Unicode and \xa0

Why \xa0 breaks your text, and how Unicode NFC, NFD, NFKD, and NFKC normalization fix encoding issues in Python.

Python Unicode Text Processing Data Cleaning

December 22, 2025 Read →

Technical Note

FlashAttention: An Advancement in GPU Acceleration for Training LLMs

Part 2 of a FlashAttention series: what FlashAttention is, how kernel fusion cuts memory traffic, and its effect on LLM training speed and quality.

Deep Learning GPU Acceleration Large Language Models Transformers

October 24, 2023 Read →

Technical Note

A Deep Dive into FlashAttention V1 -part 3

Part 3 of a FlashAttention series: the V1 algorithm in detail, covering tiling, safe softmax, and the online normalizer calculation that makes it work.

Transformers FlashAttention Large Language Models Deep Learning

October 24, 2023 Read →

Technical Note

FlashAttention: Understanding GPU Architecture-Part 1

Part 1 of a FlashAttention series: how data moves through a system, the GPU memory hierarchy, and why attention is bottlenecked by memory bandwidth.

GPU Computing Large Language Models GPU Architecture FlashAttention

October 23, 2023 Read →

credit: https://alaaalatif.github.io/2019-04-11-gelu/

Technical Note

Key takeaways from building multiple machine learning models

Practical lessons from building ML and deep learning models: visualizing embeddings with t-SNE and UMAP, weight initialization, and why data wins.

NLP Machine Learning Optimization Deep Learning

March 19, 2022 Read →