Writing log70 posts286 tagslatest Jul 24, 2026
Llm.
Latest posts
Hallmark Architecture: How Do You Stop Designs From Looking 'AI-Generated' — With a Prompt?
Hallmark is Together AI’s anti-AI-slop design skill. It ships almost no executable code — its entire intelligence lives in 106 markdown files. This piece dissects the SKILL.md router and its progressive context loading, the narrowing decision pipeline, and — at its center — the 58 slop-test gates, contrasting it with skill-injecting projects like Superpowers and SkillSpector.
Llama 3 (2024) Paper Notes
Paper notes on The Llama 3 Herd of Models: 15T tokens, a 405B flagship, 128K context, and a design philosophy of 'managing complexity' (dense over MoE, SFT → rejection sampling → DPO instead of PPO) — the paper where an open-weight model first reaches GPT-4-class performance. It completes Llama 2's skeleton through scale, data, and simplicity.
Llama 2 (2023) Paper Notes
Paper notes on Llama 2 covering its commercial license, a base model scaled to 2T tokens with 4k context and GQA, and the Llama 2-Chat alignment pipeline (SFT → two reward models → iterative RLHF with rejection sampling + PPO → GAtt). It fuses LLaMA's open base lineage with InstructGPT's RLHF at production quality.
LLaMA (2023) Paper Notes
Paper notes on LLaMA covering its core strategy — train smaller models for longer on publicly available data, optimized for inference — the result where a 13B model beats 175B GPT-3, the RMSNorm/SwiGLU/RoPE architecture changes, the 2048-A100 training scale, and how it kicked off the open LLM ecosystem.
Analyzing Firecrawl: How Do You Build an API That Turns the Web Into LLM-Ready Markdown?
Firecrawl is a data API that turns any website into clean, LLM-ready Markdown or structured JSON. It orchestrates in TypeScript, cleans in Rust, converts to Markdown in Go, races several scrape engines in a waterfall, and drives crawls with a hand-built Postgres queue. We analyze the scrape/crawl/map/search/extract pipelines and contrast them with Browser Use as "two ways to feed the web to LLMs."
Analyzing Browser Use: How Do You Show a Web Page to an LLM So It Can Drive a Browser?
Browser Use is a Python agent in which an LLM drives a real browser. It pre-chews the page into an indexed list of interactive elements for the LLM, drives the browser via CDP instead of Playwright, and runs the session with an event bus and watchdogs. We look at why this fits LLMs better, picking up from the earlier Playwright analysis.
InstructGPT (2022) Paper Notes
Paper notes on InstructGPT covering its core method — the three-step RLHF recipe (SFT → reward model → PPO) — the alignment result where a 1.3B model beats 175B GPT-3, gains in truthfulness and toxicity, the alignment tax and PPO-ptx, and the limitations.
GPT-3 (2020) Paper Notes
Paper notes on GPT-3 covering its core ideas: 175B scaling, in-context learning (zero/one/few-shot), weighted-sampling training data, headline benchmark numbers, and the data-contamination and bias limitations.
GPT-2 (2019) Paper Notes
Paper notes on GPT-2 covering its core ideas: decoder-only Transformer scaling, WebText, next-token prediction, zero-shot task transfer, and the staged release controversy.
Hermes Agent Architecture: An AI Agent That Learns on Its Own, Lives in Messengers, and Uses Tools
An architecture deep-dive into Nous Research's Hermes Agent from a user perspective. Covers the CLI, messenger gateway, ACP, tool registry, skills, memory, plugins, and sub-agent structure in an accessible, flow-oriented way.
BERT (2018) Paper Notes
Paper notes covering the core ideas of BERT: the bidirectional Transformer encoder, masked language model, next sentence prediction, and the fine-tuning paradigm.
LLM Architecture Basics: Encoder-only vs. Decoder-only
A comparison of encoder-only and decoder-only architectures that distinguish the BERT and GPT families.
LLM Basics: RNNs and Sequential Processing
Explains how RNNs — the dominant architecture before Transformers — process sequences token by token, and the fundamental limitations that motivated moving beyond them.
LLM Learning Basics: Masked Language Model
Explains BERT's core training objective — the Masked Language Model — with formulas, commentary, and examples.
LLM Learning Basics: Pre-training and Fine-tuning
Explains large-scale pre-training and task-specific fine-tuning through the lens of the BERT workflow.
Transformer Basics: Encoder and Decoder
A clear explanation of what the Transformer encoder and decoder each do, grounded in the original architecture and illustrated with simple examples.
Attention Is All You Need (2017) — Paper Notes
A reading note on the Transformer paper — the core ideas, why it mattered, and what to read next.
LLM Learning Basics: cross-entropy and perplexity
Explains cross-entropy and perplexity — the metrics used to measure how wrong a model is — with formulas, commentary, and examples.
LLM Math Basics 2: Softmax and Probability Interpretation
A walkthrough of how softmax converts raw scores into probability-like values, with formulas, explanations, and examples.
LLM Math Basics 1: Vectors and the Dot Product
A walkthrough of vectors and the dot product — with notation, explanations, and examples — covering what you need to know before reading LLM papers.
LLM Paper Evolution Map (MOC)
A Map of Content page for reading core LLM papers in order, starting from the Transformer.
Transformer Basics: Intuition Behind Q, K, and V
Explains the role of Q, K, and V through the attention formula, a plain-language walkthrough, and a concrete example.
Transformer Basics: Residual, LayerNorm, FFN
Why Residual, LayerNorm, and FFN are necessary in a Transformer block — explained with equations, commentary, and examples.
Ollama Architecture Analysis / Real-World Q&A
A comprehensive architecture analysis of Ollama, a local LLM execution platform implemented in Go. Covers the tech stack, scheduler, model management, llama.cpp integration, and the Runner system in detail.
LangChain Python Monorepo Architecture Analysis Report
A comprehensive architectural analysis of LangChain, the Python-based LLM application framework. Covers the full technology stack, core modules, the Runnable interface, the callback system, and partner package structure in detail.