# Timing LLM inference in Python without inventing a CUDA speedup — executable example

Five tests passed, including real CPU generation with a tiny random GPT-2 model. No trained-model benchmark, GPU execution or speedup measurement.

Environment: Python3.11.5; macOS14.5 arm64; torch2.9.1; transformers4.32.1; tokenizers0.13.2; numpy2.2.6; CPU.

Keep these files together: benchmark.py, test_benchmark.py. Download all at [/assets/examples/archive-accelerating-large-language-models-with-cuda-and-python/example.zip](https://oneruby.dev/assets/examples/archive-accelerating-large-language-models-with-cuda-and-python/example.zip).

## Run

The recorded commands used the interpreter paths shown below. On your computer use Python with the recorded versions and dependencies.

```sh
/Users/alexandrkholodniak/anaconda3/bin/python3 -B -m unittest -v test_benchmark.py
/Users/alexandrkholodniak/anaconda3/bin/python3 -B benchmark.py --device cpu
```


Direct dependency versions are recorded in requirements.txt. The existing installed environment was executed; a fresh dependency installation was not rerun. For the GPT-2 fixture set HF_HOME to a writable cache and TRANSFORMERS_OFFLINE=1; it loads no remote checkpoint.

## Limits

- Random 16864-parameter offline GPT-2 fixture, not a trained LLM or quality evaluation.
- CPU execution only; CUDA branch not run on hardware.
- Generation-call timing excludes loading, request transfer, tokenization, decoding and serving.
- Existing PyTorch2.9.1/Transformers4.32.1 environment tested; new installation not validated.