GPT-3 (2020) Paper Notes
Paper notes on GPT-3 covering its core ideas: 175B scaling, in-context learning (zero/one/few-shot), weighted-sampling training data, headline benchmark numbers, and the data-contamination and bias limitations.
Llm
GPT-2 (2019) Paper Notes
Paper notes on GPT-2 covering its core ideas: decoder-only Transformer scaling, WebText, next-token prediction, zero-shot task transfer, and the staged release controversy.
Hermes Agent Architecture: An AI Agent That Learns on Its Own, Lives in Messengers, and Uses Tools
An architecture deep-dive into Nous Research's Hermes Agent from a user perspective. Covers the CLI, messenger gateway, ACP, tool registry, skills, memory, plugins, and sub-agent structure in an accessible, flow-oriented way.
BERT (2018) Paper Notes
Paper notes covering the core ideas of BERT: the bidirectional Transformer encoder, masked language model, next sentence prediction, and the fine-tuning paradigm.
LLM Architecture Basics: Encoder-only vs. Decoder-only
A comparison of encoder-only and decoder-only architectures that distinguish the BERT and GPT families.
LLM Basics: RNNs and Sequential Processing
Explains how RNNs — the dominant architecture before Transformers — process sequences token by token, and the fundamental limitations that motivated moving beyond them.
LLM Learning Basics: Masked Language Model
Explains BERT's core training objective — the Masked Language Model — with formulas, commentary, and examples.
LLM Learning Basics: Pre-training and Fine-tuning
Explains large-scale pre-training and task-specific fine-tuning through the lens of the BERT workflow.
Transformer Basics: Encoder and Decoder
A clear explanation of what the Transformer encoder and decoder each do, grounded in the original architecture and illustrated with simple examples.
Attention Is All You Need (2017) — Paper Notes
A reading note on the Transformer paper — the core ideas, why it mattered, and what to read next.
LLM Learning Basics: cross-entropy and perplexity
Explains cross-entropy and perplexity — the metrics used to measure how wrong a model is — with formulas, commentary, and examples.
LLM Math Basics 2: Softmax and Probability Interpretation
A walkthrough of how softmax converts raw scores into probability-like values, with formulas, explanations, and examples.
LLM Math Basics 1: Vectors and the Dot Product
A walkthrough of vectors and the dot product — with notation, explanations, and examples — covering what you need to know before reading LLM papers.
LLM Paper Evolution Map (MOC)
A Map of Content page for reading core LLM papers in order, starting from the Transformer.
Transformer Basics: Intuition Behind Q, K, and V
Explains the role of Q, K, and V through the attention formula, a plain-language walkthrough, and a concrete example.
Transformer Basics: Residual, LayerNorm, FFN
Why Residual, LayerNorm, and FFN are necessary in a Transformer block — explained with equations, commentary, and examples.
Ollama Architecture Analysis / Real-World Q&A
A comprehensive architecture analysis of Ollama, a local LLM execution platform implemented in Go. Covers the tech stack, scheduler, model management, llama.cpp integration, and the Runner system in detail.
LangChain Python Monorepo Architecture Analysis Report
A comprehensive architectural analysis of LangChain, the Python-based LLM application framework. Covers the full technology stack, core modules, the Runnable interface, the callback system, and partner package structure in detail.