Upload README.md with huggingface_hub

README.md

@@ -13,41 +13,48 @@ library_name: pytorch
 
 # KB-Diffusion Model B — word-level masked diffusion
 
-Masked diffusion language models trained on 8,000 five-letter English
-words. The "generalization companion" experiment from
+Masked diffusion language models trained on English words. The
+"generalization companion" experiment from
 [KB-Diffusion](https://github.com/PastelRuntime/KB-Diffusion-Optimized)
 (an educational masked-diffusion project by Bijan Bowen / OminousIndustries):
 swap the repo's four keyboard layouts for thousands of words and see if the
 same recipe still works. It does — and iterating on decoding strategy turned
-out to matter more than more parameters.
+out to matter as much as architecture, with sequence length flipping which
+sampler wins.
 
-Two checkpoints, same LLaDA-style recipe (t ~ U(0.05, 1) masking, 1/t-weighted
+Three checkpoints, same LLaDA-style recipe (t ~ U(0.05, 1) masking, 1/t-weighted
 CE, bidirectional transformer, no causal mask):
 
-| | v2 | v3 |
-|---|---|---|
-| Params | 4.75M (6 layers) | 6.33M (8 layers) |
-| Steps | 8,000 | 12,000 + cosine LR |
-| Best valid English | 95.5% | **98.4%** |
-| Unique words / 512 | 409 | **428** |
-| Unigram TV vs exact Bayes | 0.0374 | **0.0135** |
+| | v2 (N=5) | v3 (N=5) | N=10 |
+|---|---|---|---|
+| Params | 4.75M (6 layers) | 6.33M (8 layers) | 6.34M (8 layers) |
+| Steps | 8,000 | 12,000 + cosine LR | 12,000 + cosine LR |
+| Best valid English | 95.5% (T=0.5) | **98.4%** (T=0.5) | 74.2% (revision + T=0.5) |
+| Unique words / 512 | 409 | **428** | 200 |
+| Unigram TV vs exact Bayes | 0.0374 | **0.0135** | 0.019 |
 
-## The headline finding
+## The headline findings
 
-Identical v2 weights, different decoding:
+**1. Temperature is the free win** (v2, frozen weights): ancestral sampling
+at T=1.0 gives 68.8% valid English; T=0.5 gives 95.5%. Same weights, same
+5 forward passes, +27 points.
 
-| Decoding | Valid English | Cost (passes) |
-|---|---|---|
-| Ancestral, temperature 1.0 | 68.8% | 5 |
-| Ancestral, temperature 0.5 | **95.5%** | 5 |
-| Revision-capable (re-mask weak commits) | 77.1% | 22.5 |
+**2. Decoding strategy > extra parameters**: v2 read well (95.5%) beats v3
+read poorly (82.6% at T=1.0).
 
-A 27-point validity jump from temperature alone, at zero extra compute. The
-sampler is half the model — twice over.
+**3. Sequence length flips the sampler winner.** At N=5, revision-capable
+sampling (un-commit weak letters, re-mask, retry) *loses* to plain
+low-temperature sampling (77.1% vs 95.5%). At N=10 it *wins* (74.2% vs
+63.3%) — early mistakes poison enough downstream positions that
+un-committing them pays for its 4x compute. The "diffusion can revise"
+capability has a measured regime where it wins.
 
-Full methodology, negative results (revision sampling not worth it at N=5;
-rare-prefix Bayes tracking degrades), prompting study, and per-sampler
-tables: see `docs/model-b.md` in the GitHub repo.
+**4. The parallel/iterative gap explodes with length**: one-shot sampling
+falls 2.0% (N=5) → 0.0% of 512 samples (N=10). This is why real diffusion
+LMs commit few tokens at a time.
+
+Full methodology, negative results, and per-sampler tables: `docs/model-b.md`
+in the GitHub repo.
 
 ## Usage
 
@@ -55,17 +62,17 @@ tables: see `docs/model-b.md` in the GitHub repo.
 import torch
 from model_b_word_diffusion_v3 import Net, CH, MASK, N  # from the GitHub repo
 
-model = Net(layers=8)  # v3
+model = Net(layers=8)  # v3; use model_b_word_diffusion_n10.py for N=10
 sd = torch.load("modelb_v3.pt", map_location="cpu", weights_only=True)
 model.load_state_dict(sd)
 model.eval()
-# ancestral confidence-commit sampler, temperature 0.5 — see repo script
+# ancestral confidence-commit sampler, temperature 0.5 — see repo scripts
 ```
 
 ## Intended use & limitations
 
-Educational artifact, not a production model: 27-token vocab, 5-position
+Educational artifact, not a production model: 27-token vocab, 5/10-position
 sequences. It exists to make the masked-diffusion mechanism (parallel
-prediction, confidence commits, re-masking, posterior sharpening)
-measurable — and to show how much decoding strategy contributes to
-generation quality on frozen weights.
+prediction, confidence commits, re-masking, revision, posterior sharpening)
+measurable — and to map how decoding strategy and sequence length interact
+on frozen weights.