folio

Training a small language model on the KJV

i decided to train & use a small language model from scratch to learn how the process actually works. i worked with a slew of frontier & open models to both build it and explain it so i could grasp it. and then we documented it.

here's how it went.

12artifacts

  1. 01the process in outlineartifact · Aug 26, 2026
  2. 02the task and the textartifact · Aug 26, 2026
  3. 03the transformerartifact · Aug 26, 2026
  4. 04interpretationartifact · Aug 26, 2026
  5. 05trainingartifact · Aug 26, 2026
  6. 06data and scaleartifact · Aug 26, 2026
  7. 07inferenceartifact · Aug 26, 2026
  8. 08transformer variantsartifact · Aug 26, 2026
  9. 09evaluationartifact · Aug 26, 2026
  10. 10post-trainingartifact · Aug 26, 2026
  11. 11the entire modelartifact · Aug 26, 2026
  12. 12from model to systemartifact · Aug 26, 2026

search the public archive.