add/update model card with eval metrics

README.md

@@ -14,13 +14,17 @@ tags:
 
 # Prompt Injection Detection (encoder, multi-label)
 
-Multi-label classifier over 9 prompt-injection attack categories,
-fine-tuned from **[`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base)**. Single
-forward pass; `is_valid` = any attack above threshold (0.5).
+Encoder classifier that detects which prompt-injection attack categories (out of
+9) appear in an input. Fine-tuned from
+**[`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base)**.
+Replaces the 2B Qwen decoder LoRA with a single-forward-pass encoder for
+lower-latency runtime-security use in LLM-Guard's `PromptInjection` scanner.
 
 - **Base model**: [`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base)
-- **Trained with**: max_seq_length=3072, epochs=10, lr=3e-05
 - **Labels (9)**: DirectInjection, Jailbreak, Adversarial, Extraction, Encoding, Manipulation, Smuggling, Indirect, MultiTurn
+- **Output**: per-category sigmoid; a category fires when its score ≥ its per-class threshold; `is_valid` = `max(score) ≥ 0.05`.
+- **Multilingual / long context**: inherited from the base encoder; trained with
+  inputs up to the base model's positional limit.
 
 ## Usage
 
@@ -52,4 +56,51 @@ result = {"is_valid": is_valid, "category": {k: True for k in present}}
 print(result)   # e.g. {"is_valid": True, "category": {"DirectInjection": True}}
 ```
 
-> Test-set metrics are added by `eval_and_push_card.py` after evaluation.
+## Decision thresholds
+
+Fitted on a held-out split (NOT the test set reported below) and stored in
+`config.json` (`category_thresholds`, `is_valid_threshold`) + `thresholds.json`.
+The Usage snippet reads them automatically — a flat 0.5 cutoff under-detects the
+imbalanced minority categories.
+
+- **`is_valid` (attack) gate**: `max(score) ≥ 0.05`
+
+| Category | threshold |
+|----------|-----------|
+| `DirectInjection` | 0.55 |
+| `Jailbreak` | 0.05 |
+| `Adversarial` | 0.45 |
+| `Extraction` | 0.55 |
+| `Encoding` | 0.45 |
+| `Manipulation` | 0.25 |
+| `Smuggling` | 0.65 |
+| `Indirect` | 0.25 |
+| `MultiTurn` | 0.70 |
+
+## Test-set metrics (n=500)
+
+| Metric | Value |
+|--------|-------|
+| is_valid (attack-detection) accuracy | 0.968 |
+| category-set (exact) accuracy | 0.688 |
+| micro-F1 | 0.789 |
+| macro-F1 | 0.785 |
+| latency mean (ms/example) | 1.7930222675204277 |
+| latency p95 (ms/example) | 1.8397919833660126 |
+| device | cuda:0 |
+
+### Per-category F1
+
+| Category | F1 | Description |
+|----------|----|-------------|
+| `Adversarial` | 0.855 | Carefully crafted inputs that exploit model quirks or training artifacts to elicit unintended behavior without an obvious override. |
+| `DirectInjection` | 0.824 | Explicit instruction overrides that tell the model to ignore prior context (e.g. "ignore all previous instructions and …"). |
+| `Encoding` | 0.752 | Obfuscated payloads using base64 / ROT13 / leetspeak / homoglyphs / zero-width chars / shell pipes to bypass keyword filters. |
+| `Extraction` | 0.765 | Attempts to leak the system prompt, hidden instructions, or memorized training data (e.g. "print everything between <<system>> tags"). |
+| `Indirect` | 0.838 | Injection delivered through untrusted retrieved content (RAG passages, scraped pages, file contents) rather than the user's direct turn. |
+| `Jailbreak` | 0.737 | Persona / role swaps and constraint bypasses aimed at disabling safety alignment (e.g. DAN, "you are now an unrestricted assistant"). |
+| `Manipulation` | 0.679 | Social-engineering framings (urgency, authority, sympathy, false context) that pressure the model into compliance. |
+| `MultiTurn` | 0.689 | Crescendo / drip-feed attacks that build up across multiple turns to gradually erode guardrails. |
+| `Smuggling` | 0.926 | Hidden control tokens, chat-template markers, or special sequences injected to confuse the parser (e.g. fake `<|im_end|>` / role tags). |
+
+*Evaluated on `test_dataset_injection.csv`. Generated 2026-06-03 18:59 UTC.*