marin-community/delphi-1e23-25Bparams-628Btokens

Text Generation • 25B • Updated 2 days ago • 293 • 3

Note Held-out predicted optima. The Delphi scaling law was fit on the seven IsoFLOP-bucket winners further down this collection, then used to forecast loss at three held-out compute budgets — 1e21, 1e22, and 1e23 FLOPs — before the runs at those budgets were trained. This 25 B-parameter run is the headline 1e23 forecast target. The pre-registered forecast landed within 0.2% of the observed loss, using ~300× less compute than this training run itself.

marin-community/delphi-1e22-9.7Bparams-160Btokens

Text Generation • 10B • Updated 2 days ago • 414

Note 9.7 B parameters at 1 × 10²² FLOPs. The pre-registered forecast lands +0.2% of observed.

marin-community/delphi-1e22-9.7Bparams-160Btokens-seed42

Text Generation • 10B • Updated 2 days ago • 47

Note Same architecture and recipe as the canonical 1e22 run; only the RNG seed differs (42).

marin-community/delphi-1e22-9.7Bparams-160Btokens-seed62746

Text Generation • 10B • Updated 2 days ago • 47

Note Same architecture and recipe; RNG seed = 62746. Together with seed 42, the seed spread is ~0.1% — about 10× tighter than the bootstrap CI on the scaling-law forecast.

marin-community/delphi-1e21-3.4Bparams-46.3Btokens

Text Generation • 3B • Updated 2 days ago • 126

Note 3.4 B parameters at 1 × 10²¹ FLOPs. The pre-registered forecast lands +0.5% of observed.

marin-community/delphi-1e21-3.4Bparams-46.3Btokens-seed42

Text Generation • 3B • Updated 2 days ago • 55

Note Same architecture and recipe as the canonical 1e21 run; RNG seed = 42.

marin-community/delphi-1e21-3.4Bparams-46.3Btokens-seed62746

Text Generation • 3B • Updated 2 days ago • 53

Note Same architecture and recipe; RNG seed = 62746.

Note Plot-ready data behind every figure in the Delphi blog post — one config per figure, with a wandb_url on every row so you can click through to the training curves. IsoFLOP-bucket winners — the fit data. At each of seven compute budgets from 3 × 10¹⁸ to 3 × 10²⁰ FLOPs, several (model size, token count) shapes were trained at the same total compute. The lowest-loss shape per bucket is the empirical compute-optimum for that budget; the Delphi scaling law is the asymptotic power-law fit through

marin-community/delphi-3e18-447Mparams-1.2Btokens

Text Generation • 0.4B • Updated 2 days ago • 296

Note Other IsoFLOP grid points. The remaining 74 sweep runs trace the IsoFLOP parabolas — every (model size, token count) shape Delphi trained at each compute budget, sorted by FLOP budget descending then by parameter count descending. Useful for reproducing the IsoFLOP fits, studying off-optimal training, or fitting your own scaling law on the same recipe and data.