LLaMA (2023) Paper Notes
Paper notes on LLaMA covering its core strategy — train smaller models for longer on publicly available data, optimized for inference — the result where a 13B model beats 175B GPT-3, the RMSNorm/SwiGLU/RoPE architecture changes, the 2048-A100 training scale, and how it kicked off the open LLM ecosystem.