Choose nanochat to own the training pipeline
If you want to change how a small language model is trained, nanochat puts the relevant stages in one codebase. Its README describes tokenization, pretraining, fine-tuning, evaluation and inference. That breadth is useful when your question crosses stages, such as whether a tokenizer change helps the model you eventually chat with.
The practical commitment is to run and understand that pipeline. Nanochat's reference speedrun targets an eight-H100 GPU node. Our small CPU sandbox never established that training route. It also hit a missing import during the test command, so a successful installation alone is an especially poor shortcut for deciding this project is ready on your machine.
What happened when we ran it
Our nanochat test command exited 1 in 4 seconds, with 14 passed tests, zero failed cases and five collection/setup errors out of 19 recorded entries. The November 2, 2026 run used commit 92d63d4. Installation succeeded in 18 seconds and the build step succeeded in 2 seconds, but those outcomes did not produce a passing test run.
The failing log shows ModuleNotFoundError: No module named 'rustbpe' while importing nanochat/tokenizer.py. Its summary names errors in the attention fallback, engine, optimizer, tasks and tokenizer test modules. The supplied tail does not establish why the import was unavailable or whether correcting it would expose further problems. Calling the project broken would go beyond this evidence. Calling these tests successful would contradict it.
The record lists 36 dependencies, 36 MB on disk and 17 known vulnerabilities from pip-audit. It doesn't identify the advisory severities or affected packages. Nor is the recorded footprint a measurement of downloaded datasets, model checkpoints or the complete GPU training environment. Inspect the advisories and the selected environment before using this checkout for work that matters.
The sandbox configuration specified 3 CPUs, 8 GB RAM and image ghcr.io/astral-sh/uv:python3.12-bookworm. Actual runtime versions require raw logs. The checkout had 53 files, approximately 8,199 source lines and occupied 1.1 MB. The scan recorded a tests directory, zero CI workflow files and no Dockerfile. We measured no model quality, training throughput or GPU operation.
CPU examples shrink the goal as well as the model
The README provides separate uv extras for GPU and CPU/MPS use, plus a development dependency group. Its CPU/MPS example explicitly shrinks the model and warns that it won't give strong results. That's an appropriate route for inspecting the training machinery. It doesn't make a laptop a substitute for the reference GPU run.
For GPUs with less memory, the README advises reducing device batch size. It also qualifies less-exercised hardware paths. A developer choosing nanochat should therefore be comfortable matching the script to available hardware and distinguishing an out-of-memory constraint from a modeling problem. Our CPU record cannot predict which configuration fits your accelerator.
The project's depth option controls model size and derives other training settings. That gives experiments a common starting convention. If you change the model or training procedure, examine those derived settings rather than assuming the convenient control proves the new configuration is optimal. The upstream description of compute-optimal scaling is a design claim, not a result our sandbox verified.
Data downloads and tracking belong in the run plan
The reference script downloads dataset shards, trains and evaluates the tokenizer, then runs pretraining and supervised fine-tuning with evaluation. Its default artifact directory is under the user's cache. Allow for the data and checkpoint lifecycle when choosing storage. The recorded installation footprint doesn't cover that lifecycle.
Weights & Biases logging is optional in the reference script. The comments describe logging in and selecting a run name to enable it; the default dummy run skips that logging. Decide whether you want the hosted tracking service before launching. Any cloud GPU credentials belong to your chosen provider, and our run didn't validate a provider account, a training job or a conversation with a trained model.
TRL fits post-training; nanoGPT is an older study option
Hugging Face TRL documents trainers for supervised fine-tuning and preference optimization. Evaluate it when your starting point is an existing foundation model and the work is post-training. That avoids choosing a tokenizer-to-chat training harness for a task centered on adapting weights you already have. We have no comparative runtime or model-quality result for TRL.
nanoGPT still exposes an older GPT model and training loop for study, but its README marks the repository deprecated and directs new users to nanochat. Consider it only when that historical, narrower implementation is what you want to inspect. It would be a poor default replacement for someone seeking current development.
Recent proposals don't prove a supported deployment
The repository metadata gives September 7, 2026 as the last push. The issues-and-PRs feed nevertheless shows new PR activity on November 2, including training-resume and checkpoint-discovery proposals. Read those together: there is current contributor activity, but an open proposal is not a shipped fix. This evidence establishes neither abandonment nor a support response-time promise.
Documentation receives 4 on an editorial scale where 1 means instructions obstruct evaluation, 3 means a usable route with material gaps, and 5 means the documented routes have been independently checked; 2 and 4 sit between those anchors. The README and reference script explain hardware choices and stages, while our run verifies none of the full training routes. Setup, community support and overall maturity remain unassessed. Before renting a training node, make the tokenizer import work in the environment you intend to use and rerun the affected checks.