Train your first
decoder-only model
from scratch.
Not a fine-tune. Not an adapter. Random weights, a real corpus, and a real loss curve you watch fall in real time — from a browser tab, on our GPUs.
Signing in opens NanoDex full screen — Hugging Face's login page can't run inside an embedded frame.
Pick a size
Four architectures, from 492k to 8.1M parameters. That's the real total — embeddings included, no asterisk.
Pick a token budget
Anywhere from 1M to 1.5 billion tokens of fineweb-edu. More tokens, sharper model, longer wait.
Watch it learn
Your run joins the queue. Live loss curve, throughput, logs — then the weights land in your own Hugging Face account.
Small enough to finish. Big enough to learn something.
2 heads (1 KV) · FFN 147
4 heads (2 KV) · FFN 215
5 heads (1 KV) · FFN 285
6 heads (2 KV) · FFN 472
8 heads (2 KV) · FFN 647
8 heads (2 KV) · FFN 839
8 heads (2 KV) · FFN 1020
10 heads (2 KV) · FFN 1856
8 heads (2 KV) · FFN 2669
8 heads (2 KV) · FFN 1792
12 heads (12 KV) · FFN 3072
A model that is genuinely yours
Every finished run is pushed to your Hugging Face namespace: config.json, model.safetensors, the tokenizer, a training_run.json with the full recipe, and a generated model card. Load it with transformers like anything else on the Hub.
It will not be an assistant
At a few hundred thousand parameters, a model learns word shapes, common collocations and a little syntax. It will produce English-looking text with no facts in it. That's the correct outcome — and watching cross-entropy fall from 7.6 (uniform noise over 2,048 tokens) to somewhere near 4 is the whole point.
The newest models on the shelf
Ready when you are.
Sign in with Hugging Face — no separate account, no password.