Writing log70 posts286 tagslatest Jul 13, 2026
Llama-2.
Latest posts
Llama 2 (2023) Paper Notes
Paper notes on Llama 2 covering its commercial license, a base model scaled to 2T tokens with 4k context and GQA, and the Llama 2-Chat alignment pipeline (SFT → two reward models → iterative RLHF with rejection sampling + PPO → GAtt). It fuses LLaMA's open base lineage with InstructGPT's RLHF at production quality.