Training a small language model on the KJV
i decided to train & use a small language model from scratch to learn how the process actually works. i worked with a slew of frontier & open models to both build it and explain it so i could grasp it. and then we documented it.
here's how it went.
12artifacts
01the process in outlineartifact · Aug 26, 2026
02the task and the textartifact · Aug 26, 2026
03the transformerartifact · Aug 26, 2026
04interpretationartifact · Aug 26, 2026
05trainingartifact · Aug 26, 2026
06data and scaleartifact · Aug 26, 2026
07inferenceartifact · Aug 26, 2026
08transformer variantsartifact · Aug 26, 2026
09evaluationartifact · Aug 26, 2026
10post-trainingartifact · Aug 26, 2026
11the entire modelartifact · Aug 26, 2026
12from model to systemartifact · Aug 26, 2026