Writing log71 posts289 tagslatest Jul 24, 2026
Alignment.
Latest posts
Llama 3 (2024) Paper Notes
Paper notes on The Llama 3 Herd of Models: 15T tokens, a 405B flagship, 128K context, and a design philosophy of 'managing complexity' (dense over MoE, SFT → rejection sampling → DPO instead of PPO) — the paper where an open-weight model first reaches GPT-4-class performance. It completes Llama 2's skeleton through scale, data, and simplicity.
Llama 2 (2023) Paper Notes
Paper notes on Llama 2 covering its commercial license, a base model scaled to 2T tokens with 4k context and GQA, and the Llama 2-Chat alignment pipeline (SFT → two reward models → iterative RLHF with rejection sampling + PPO → GAtt). It fuses LLaMA's open base lineage with InstructGPT's RLHF at production quality.
InstructGPT (2022) Paper Notes
Paper notes on InstructGPT covering its core method — the three-step RLHF recipe (SFT → reward model → PPO) — the alignment result where a 1.3B model beats 175B GPT-3, gains in truthfulness and toxicity, the alignment tax and PPO-ptx, and the limitations.