# How to Train an LLM: Verify the Training Step Before Scaling It: runnable files

5 CPU tests with Transformers 4.56.2 and PyTorch 2.9.1; random tiny GPT-2 training and local reload, no pretrained checkpoint, GPU or useful-language-model claim.

Download all files in this folder before running. The article is at https://oneruby.dev/how-to-train-a-language-learning-model-llm/.

## Reproduce

Tested with Python 3.11.5 on arm64 macOS. Use an isolated environment; requirements.txt pins direct dependencies and requirements.lock records the tested dependency closure for this platform.

```sh
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.lock
.venv/bin/python -B -m unittest -v test_example.py
.venv/bin/python -B example.py
```

PyTorch was already installed on this machine and reused through a temporary venv with system-site-packages; Transformers was installed in that venv. No model is downloaded: GPT2Config creates random weights, and from_pretrained reloads only a temporary locally produced directory. For strict offline model loading set HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. The lock records macOS dependencies; choose a platform-appropriate PyTorch distribution elsewhere.

## Files

- [requirements.lock](requirements.lock)

- [example.py](example.py)
- [requirements.txt](requirements.txt)
- [test_example.py](test_example.py)

## Boundaries

- Tiny random GPT-2 architecture with a nine-token fixed vocabulary; not pretrained fine-tuning or a useful translator.
- CPU only; no CUDA, GPU memory measurement, pretrained model download or remote inference.
- Two training phrases and one held-out phrase illustrate overfitting, not statistically reliable model quality.
- Reload checks model weights and logits; optimizer state and training resume are not verified.