textilelabs/Loom-Router-1

Loom Router 1

1.4M parameters · 2.8MB · Textile Labs

Give it a user message. It tells you which tool should handle it, in one token.

That's the whole product. Your harness passes the user's original text to whichever tool it names — the model never rewrites your input, so nothing can be copied wrong or malformed.

"whats the weather in leeds tomorrow"  →  <route:weather>
"remind me to call mum at 6"           →  <route:reminder>
"whats my sisters name"                →  <route:unknowable>

86.5% accuracy on 2,969 real held-out human utterances, across 17 routes. Random guessing scores 5.9%.

It is trained from scratch — randomly initialised weights, trained end to end. Nothing is fine-tuned from a pretrained base. Comparable open routers we looked at are considerably larger and fine-tuned from pretrained checkpoints; we make no claim to be the smallest of its kind.

What it's for

A first stage in front of a bigger model or an agent loop. Deciding which tool to reach for is a cheap decision that does not need a large model — but people usually pay for a large model to make it. This does it in one token, on a CPU, in a 2.8MB file.

Concretely: use it to pick the tool, then hand the user's original text to that tool. Or use it to decide whether you need to call a large model at all.

It is not a chat model. It has no conversational output and cannot introduce itself. It answers with a route and nothing else.

The routes

Tools (13) — search calc time weather calendar reminder email notes maps translate convert define music

Control (4) — answer clarify unknowable refuse

The Loom philosophy, as routes

Every Loom model is built on the same bet: at small sizes, knowing your limits is more achievable than knowing things — and more useful. In a generative model that means saying "I don't know". In a router it becomes something sharper — a decision:

route	what it means
answer	no tool needed. Don't reach for one reflexively.
clarify	the request is ambiguous. Don't guess — ask.
unknowable	this depends on something only the user knows. No tool can fix that.
refuse	this shouldn't be done.

A router that only answers "which tool?" has assumed a tool is always the answer. In an agent loop that assumption is the expensive one: sending "what's my sister's name" to a search tool burns a call and returns a confident wrong answer. answer and clarify are also what let a loop terminate instead of spinning.

So this card publishes the false-tool-call rate: how often it sends a request to a tool that cannot possibly help. Ours is 20.2%, and the honest reading of that is below.

Measured

Evaluated one bare prompt at a time, the way the model is actually used.

Overall 86.5% · tools 89.3% · control 71.2%

route	n	recall		route	n	recall
translate	21	100.0%		email	202	87.6%
notes	163	95.1%		reminder	134	87.3%
weather	113	93.8%		search	599	86.8%
music	368	93.8%		define	104	84.6%
answer	335	93.7%		calc	32	71.9%
convert	64	90.6%		refuse	36	25.0%
time	123	89.4%		clarify	41	12.2%
calendar	342	88.9%		unknowable	54	7.4%
maps	238	88.7%

Independent test — SNIPS

The 86.5% above is a held-out split of the same corpora used for training. To check it generalises beyond that, it was also run against SNIPS, a dataset that played no part in training at all.

73.8% on 500 unseen utterances (5 intents with an unambiguous mapping):

SNIPS intent	→ route	score
AddToPlaylist	music	91%
PlayMusic	music	87%
SearchScreeningEvent	search	80%
SearchCreativeWork	search	68%
GetWeather	weather	43%

SNIPS' BookRestaurant and RateBook have no defensible route in this ontology, so they were left unscored rather than graded against a debatable label.

The drop from 86.5% to 73.8% is the honest cost of moving to a different data distribution. GetWeather at 43% is the instructive failure: SNIPS asks about weather without using the word — "Is there a storm now in NC?", "humidity in Olvey New Hampshire", "Will there be fog…". Those go to search. The model keys on the vocabulary it was trained on, not on a general concept of weather. If your domain uses terms outside everyday assistant phrasing, expect the same and plan to retrain with them included.

Read this before relying on it

Tool routing works. The honesty routes largely do not. clarify 12.2%, unknowable 7.4%, refuse 25.0%. Treat a tool prediction as a strong signal and a control prediction as a weak hint.

The cause is understood and worth stating plainly. On synthetic data those routes scored ~76%, because "my" and "I" were reliable cues. Real assistant traffic is full of "my calendar", "my alarms", "remind me" — so the cue stopped being a cue. The real distinction is whether the referent lives in a tool's data or only in the user's head, which is a subtler thing to learn. Tripling the control training data made it worse, so it is not a volume problem.

calc (71.9%) has only 32 validation examples; that figure is noisy.

Usage — Ollama

ollama run hf.co/textilelabs/Loom-Router-1 "whats the weather in leeds tomorrow"
# <route:weather>

Ollama reads the template and params files in this repo, so there is nothing to set up. params pins temperature: 0 and num_predict: 4 — a router should be deterministic and emit one token. To build it locally instead: ollama create loom-router-1 -f Modelfile.

Usage — transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

ROUTES = ["search","calc","time","weather","calendar","reminder","email","notes",
          "maps","translate","convert","define","music","answer","clarify",
          "unknowable","refuse"]

tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Router-1")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Router-1").eval()

route_ids = {tok.convert_tokens_to_ids(f"<route:{r}>"): r for r in ROUTES}
ids_t = torch.tensor(list(route_ids))

def route(message: str) -> str:
    prompt = f"<user>\n{message.strip()}\n<|eot|>\n<loom>\n"
    ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        logits = model(input_ids=ids).logits[0, -1]
    # Decide only among legal routes, so the output is always a valid label.
    return route_ids[int(ids_t[logits[ids_t].argmax()])]

route("add milk to my shopping list")   # -> 'notes'

The prompt format is exact: <user>\n{message}\n<|eot|>\n<loom>\n, no trailing space.

In an agent loop

user → router → your harness runs the tool → result → router again
                                           → 'answer' ends the loop

Cap the number of steps in your harness. answer and clarify are the terminating routes.

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 2,048 tokens
loom-router-1-f16.gguf                    2.8MB, for Ollama / llama.cpp
template / params                         read automatically by `ollama run hf.co/...`
Modelfile                                 for building locally
ATTRIBUTION.md                            required credits for the training corpora

Training data

Real human utterances from two openly licensed corpora, remapped onto the routes above:

• MASSIVE — Amazon (CC BY 4.0), derived from SLURP (CC BY 4.0)
• CLINC150 — clinc/oos-eval (CC BY 3.0)

23,674 real utterances. The four control routes have no public equivalent and are procedurally generated. Validation is a held-out split of the real utterances — never templates written by the same process that produced the training data.

See ATTRIBUTION.md; both licences require credit.

License

Model: MIT. Training data retains its original licences and attribution.