Flag that val bpb is not comparable across the batched/varlen pipelines

README.md

@@ -49,9 +49,24 @@ DecoderStack tracks 0.2–1.4% behind through step 1000 (where the upstream base
 stopped) while running ~6% faster per step (1,212 vs 1,291 ms/step). Full head-to-head in
 [`logs/baseline_report.md`](logs/baseline_report.md).
 
+⚠️ **but val bpb is not really comparable across the two pipelines.** Upstream nanochat
+batches with best-fit cropping, discarding ~35% of tokens to pack rows exactly (its own
+dataloader docstring's figure) against DecoderStack's 11–12% `seq_len` truncation — so the
+two are not scoring the same text. And upstream measures loss with **cross-document
+attention**: its rows concatenate several documents with no isolation (`cu_seqlens` appears
+nowhere in upstream `nanochat/gpt.py`), where DecoderStack's varlen path isolates documents
+in validation as in training. The direction of that bias has not been measured. Treat the
+throughput numbers as the solid comparison and the bpb deltas as indicative.
+
 `ChrisMcCormick/nanochat-varlen-d24-2026-03-22` is the closest sibling release
-(0.724772 val bpb, 0.2614 CORE, 97.4 min) but it is **not** a controlled comparison: it
-trained with FP8 and a different 32k tokenizer.
+(0.724772 val bpb, 0.2614 CORE, 97.4 min). It is varlen-trained, so it does not carry the
+caveat above, but it is still **not** a controlled comparison: it trained with FP8 and a
+different 32k tokenizer.
+
+Both numbers also deserve error bars: Karpathy's re-runs of an identical d24 config span
+**0.0153 in CORE** (5 runs, mean 0.261), so our 0.2517 and that 0.2614 are inside each
+other's noise. Sources in
+[`stacks/models/nanochat/reference/METRICS.md`](https://github.com/chrisjmccormick/stacks/blob/main/models/nanochat/reference/METRICS.md).
 
 ## ⚠️ Tokenizer
 
@@ -208,7 +223,7 @@ a toy config — structure, per-rank sharding, shard reassembly, and a real warm
 
 Trained by [`chrisjmccormick/stacks`](https://github.com/chrisjmccormick/stacks) — the
 converter is committed at
-[`a5e608f`](https://github.com/chrisjmccormick/stacks/commit/a5e608fac7759aefd5037ca1a985d0165a7d285c)
+[`5f3d29e`](https://github.com/chrisjmccormick/stacks/commit/5f3d29e)
 (`utils/convert_ckpt_to_nanochat.py`). The training script that produced these weights is
 `code/run_full_d24_w8.py` in this repo, which is the run copy of the single-file d24
 trainer with three launcher overrides (`micro_batch_tokens` 32768→65536,