Hallmark is Together AI’s anti-AI-slop design skill. It ships almost no executable code — its entire intelligence lives in 106 markdown files. This piece dissects the SKILL.md router and its progressive context loading, the narrowing decision pipeline, and — at its center — the 58 slop-test gates, contrasting it with skill-injecting projects like Superpowers and SkillSpector.
Paper notes on The Llama 3 Herd of Models: 15T tokens, a 405B flagship, 128K context, and a design philosophy of 'managing complexity' (dense over MoE, SFT → rejection sampling → DPO instead of PPO) — the paper where an open-weight model first reaches GPT-4-class performance. It completes Llama 2's skeleton through scale, data, and simplicity.
Paper notes on Llama 2 covering its commercial license, a base model scaled to 2T tokens with 4k context and GQA, and the Llama 2-Chat alignment pipeline (SFT → two reward models → iterative RLHF with rejection sampling + PPO → GAtt). It fuses LLaMA's open base lineage with InstructGPT's RLHF at production quality.
Paper notes on LLaMA covering its core strategy — train smaller models for longer on publicly available data, optimized for inference — the result where a 13B model beats 175B GPT-3, the RMSNorm/SwiGLU/RoPE architecture changes, the 2048-A100 training scale, and how it kicked off the open LLM ecosystem.
Firecrawl is a data API that turns any website into clean, LLM-ready Markdown or structured JSON. It orchestrates in TypeScript, cleans in Rust, converts to Markdown in Go, races several scrape engines in a waterfall, and drives crawls with a hand-built Postgres queue. We analyze the scrape/crawl/map/search/extract pipelines and contrast them with Browser Use as "two ways to feed the web to LLMs."
Browser Use is a Python agent in which an LLM drives a real browser. It pre-chews the page into an indexed list of interactive elements for the LLM, drives the browser via CDP instead of Playwright, and runs the session with an event bus and watchdogs. We look at why this fits LLMs better, picking up from the earlier Playwright analysis.
Paper notes on InstructGPT covering its core method — the three-step RLHF recipe (SFT → reward model → PPO) — the alignment result where a 1.3B model beats 175B GPT-3, gains in truthfulness and toxicity, the alignment tax and PPO-ptx, and the limitations.
Paper notes on GPT-3 covering its core ideas: 175B scaling, in-context learning (zero/one/few-shot), weighted-sampling training data, headline benchmark numbers, and the data-contamination and bias limitations.
Paper notes on GPT-2 covering its core ideas: decoder-only Transformer scaling, WebText, next-token prediction, zero-shot task transfer, and the staged release controversy.
An architecture deep-dive into Nous Research's Hermes Agent from a user perspective. Covers the CLI, messenger gateway, ACP, tool registry, skills, memory, plugins, and sub-agent structure in an accessible, flow-oriented way.
Paper notes covering the core ideas of BERT: the bidirectional Transformer encoder, masked language model, next sentence prediction, and the fine-tuning paradigm.
A comparison of encoder-only and decoder-only architectures that distinguish the BERT and GPT families.
Explains how RNNs — the dominant architecture before Transformers — process sequences token by token, and the fundamental limitations that motivated moving beyond them.
Explains BERT's core training objective — the Masked Language Model — with formulas, commentary, and examples.
Explains large-scale pre-training and task-specific fine-tuning through the lens of the BERT workflow.
A clear explanation of what the Transformer encoder and decoder each do, grounded in the original architecture and illustrated with simple examples.
A reading note on the Transformer paper — the core ideas, why it mattered, and what to read next.
Explains cross-entropy and perplexity — the metrics used to measure how wrong a model is — with formulas, commentary, and examples.
A walkthrough of how softmax converts raw scores into probability-like values, with formulas, explanations, and examples.
A walkthrough of vectors and the dot product — with notation, explanations, and examples — covering what you need to know before reading LLM papers.
A Map of Content page for reading core LLM papers in order, starting from the Transformer.
Explains the role of Q, K, and V through the attention formula, a plain-language walkthrough, and a concrete example.
Why Residual, LayerNorm, and FFN are necessary in a Transformer block — explained with equations, commentary, and examples.
A comprehensive architecture analysis of Ollama, a local LLM execution platform implemented in Go. Covers the tech stack, scheduler, model management, llama.cpp integration, and the Runner system in detail.
A comprehensive architectural analysis of LangChain, the Python-based LLM application framework. Covers the full technology stack, core modules, the Runnable interface, the callback system, and partner package structure in detail.