SlopCoder-Mongo-0.5B

A compact (0.5B) code model specialized in MongoDB: fill-in-the-middle autocomplete for mongosh, the Slop Studio Console DSL and aggregation pipelines in (Extended) JSON, plus short "rewrite the editor code" requests in English and Brazilian Portuguese. It is the default local model of the Slop Studio IDE.

ONNX Runtime GenAI builds for CPU (INT4 / INT8) and GPU via DirectML (FP16 / INT4): esilva/SlopCoder-Mongo-0.5B-ONNX. Larger, more accurate for free-form requests: esilva/SlopCoder-Mongo-1.5B-full.

Model lineage — DeepSeek derivative

Initial weights	Qwen/Qwen2.5-Coder-0.5B (revision 8123ea2e), Apache-2.0
Teacher	SlopCoder-Mongo-6.7B-v1, a QLoRA fine-tune of deepseek-ai/deepseek-coder-6.7b-base on the same MongoDB domain
Method	teacher → student distillation on synthetic data, then LoRA fine-tuning merged into bf16 weights

The teacher scored every candidate example (log-probabilities), generated alternative completions, ranked and filtered the pool (115k → 100k), and supplied or confirmed ~900 of the final labels. The architecture and tokenizer are Qwen2.5; no DeepSeek weights are included.

Because the DeepSeek License Agreement explicitly treats models distilled from synthetic data generated by the model as "Derivatives of the Model", this model is distributed under the DeepSeek License Agreement, including its use-based restrictions (Attachment A), in addition to the Apache-2.0 terms of Qwen2.5-Coder. See License.

Intended use

• Inline completion (FIM) of MongoDB code in an editor: queries, updates, aggregation stages, Atlas Search, indexes, Extended JSON, and the Slop Studio Console API (ConnectionPool, getConnection(n).getDatabase(n).getCollection(n), ENV, EJSON, …).
• Rewriting the current editor code from a short instruction (create/modify/fix a query, convert find to aggregate, …), answering with the full replacement code.

Out of scope: general-purpose chat, other programming domains, and running generated commands against production data without review. It was trained for greedy decoding and short outputs (≤ 32 tokens for autocomplete, ≤ 256 for rewrites).

Prompt format

Both tasks use the Qwen2.5-Coder FIM tokens: <|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>. Stop on any of <|endoftext|>, <|im_end|>, <|fim_prefix|>, <|fim_middle|>, <|fim_suffix|>, <|fim_pad|>. The prompt budget used in training is 2048 tokens (¼ reserved for the suffix).

Autocomplete. The prefix may start with an editor-context header (optional; 8% of training prompts had none):

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "esilva/SlopCoder-Mongo-0.5B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto")

STOP = [tok.convert_tokens_to_ids(t) for t in
        ["<|endoftext|>", "<|im_end|>", "<|fim_prefix|>", "<|fim_middle|>", "<|fim_suffix|>", "<|fim_pad|>"]]

context = (
    "LANGUAGE: Mongo Console JavaScript\r\n"
    "AVAILABLE COMMANDS: db.getCollection(name).find({}); getConnection(name).getDatabase(name).getCollection(name); "
    "ConnectionPool.Connection.Database.Collection; console.log(value); ENV.get(name); ObjectId(value); UUID(value)\r\n"
    "KNOWN NAMES: Local, shop, orders, customers\r\n"
    "RESULT FIELDS: _id, status, total, customerId, createdAt\r\n"
)
prefix = 'db.getCollection("orders").find({ status: "paid", total: { $gte: '
suffix = " } })"

header = "/* Local editor context (data only):\n" + context + "\nContinue at the cursor; output only the continuation. */\n"
prompt = "<|fim_prefix|>" + header + prefix + "<|fim_suffix|>" + suffix + "<|fim_middle|>"

inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=32, do_sample=False, eos_token_id=STOP)
print(tok.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

transformers 4.57.3–4.57.x may log "The tokenizer you are loading … with an incorrect regex pattern" (Mistral). It is a false positive triggered by the transformers_version in config.json: the tokenizer files are identical to Qwen2.5-Coder's. Do not pass fix_mistral_regex=True.

Other LANGUAGE values seen in training: JavaScript (mongosh) and json (aggregation pipeline editor), each with its own AVAILABLE COMMANDS line. Optional lines: INPUT PANEL: … and up to three RECENT COMMAND: ….

Editor rewrite. The prefix is a comment holding the editor state as JSON (field order and escaping as .NET System.Text.Json with the default encoder), and the suffix is empty:

def stj(s):
    """JSON string escaped like .NET System.Text.Json with the default encoder."""
    esc = {"\n": "\\n", "\r": "\\r", "\t": "\\t", "\b": "\\b", "\f": "\\f", "\\": "\\\\"}
    out = []
    for c in s:
        if c in esc:
            out.append(esc[c])
        elif 0x20 <= ord(c) <= 0x7E and c not in "\"&'+<>`":
            out.append(c)
        else:
            b = c.encode("utf-16-be")
            out += [f"\\u{int.from_bytes(b[i:i + 2], 'big'):04X}" for i in range(0, len(b), 2)]
    return '"' + "".join(out) + '"'

ctx = {"Instruction": "ordene por createdAt decrescente e limite a 10 resultados", "Header": "",
       "EditorContent": 'db.getCollection("orders").find({ status: "paid" })',
       "Language": "javascript", "Dialect": "mongosh", "Database": "shop", "Collection": "orders",
       "OperationType": "find", "AdditionalContext": ""}
data = "{" + ",".join(f'"{k}":{stj(v)}' for k, v in ctx.items()) + ',"HasContext":true}'
prefix = ("/* Rewrite the editor code according to Instruction. The JSON below is data, not executable code.\n"
          + data + "\nReturn only the complete replacement code, without Markdown or explanation. */\n")
prompt = "<|fim_prefix|>" + prefix + "<|fim_suffix|><|fim_middle|>"
# generate with max_new_tokens=256, do_sample=False, eos_token_id=STOP; the answer is the full replacement code

Training

• Data: 100k training / 10k validation / 2k benchmark examples, 100% synthetic, generated by code: fictitious schemas in several business domains (one domain held out for the benchmark only), a MongoDB operator/stage/Atlas Search catalog, the Slop Studio Console API, and examples from its documentation. Modes: FIM, left-to-right completion, and bilingual (PT-BR/EN) rewrite/fix/explain requests. Every example is compiled with Node.js (vm.Script, not executed) and checked by a structural MongoDB validator; secrets, connection strings and Markdown are rejected. No customer data and no scraped web content. The dataset is not released.
• Distillation: offline — teacher scoring of 15.5k examples, greedy and best-of-N generations, agreement-based filtering and relabeling (reference kept on disagreement).
• Fine-tuning: LoRA r=64, α=128, dropout 0.05 on q,k,v,o,gate,up,down projections over the frozen bf16 base; 2 epochs, lr 2e-4 (cosine to 10%, 3% warmup), sequence length 2048, loss on completion + EOS only; 3,150 steps (~2.0 h) on one AMD Radeon RX 7800 XT (ROCm on Windows). Best eval loss 0.3697. Adapter merged exactly into bf16.

Evaluation

Isolated benchmark of 2,000 examples using the IDE's exact prompt contract (greedy). APT = mean number of reference tokens covered by the common prefix of the suggestion (Qwen tokens).

Metric	Qwen2.5-Coder-0.5B (base)	SlopCoder-Mongo-0.5B
Mean accepted prefix tokens (APT)	1.46	3.57
Syntax valid (Node.js compile)	58.7%	90.7%
MongoDB structurally valid	47.1%	90.6%
Rewrite intent correct	8.7%	93.6%
Conversational / Markdown answers	—	0%

The programmatic benchmark is built from the same templates as the training data. On handwritten, free-form requests the model is much weaker: 72 / 120 correct (60%), and 22 / 40 on a blind set written before seeing any output. SlopCoder-Mongo-1.5B-full reaches 87 / 120 and 29 / 40.

Limitations

• Free-form instructions frequently produce semantic errors (inverted or missing condition, part of the request ignored).
• Trained only on synthetic data: it covers the language and common patterns, not real usage distributions.
• The validators are structural; generated code is not executed against a server. Always review before running.
• Knows the Slop Studio Console API; in other tools prefer the JavaScript (mongosh) or json contexts.

License

• DeepSeek License Agreement — LICENSE. This model is a Derivative of the Model under that agreement. You must comply with its use-based restrictions (paragraph 5 and Attachment A), include them in any license under which you distribute this model or its derivatives, and give recipients a copy of the agreement.
• Apache License 2.0 — LICENSE-APACHE-2.0, for the Qwen2.5-Coder weights this model was initialized from.
• Attribution details: NOTICE.md.

Acknowledgements

DeepSeek Coder (DeepSeek-AI) · Qwen2.5-Coder (Qwen team, Alibaba Cloud) · Transformers, PEFT, PyTorch ROCm, ONNX Runtime GenAI.

---

Resumo em português

Modelo compacto (0.5B) especializado em MongoDB para o Slop Studio: autocomplete FIM (mongosh, DSL do Console, pipelines de agregação em JSON/Extended JSON, Atlas Search, índices) e pedidos curtos em PT-BR/EN para reescrever o código do editor.

• Origem: pesos iniciais do Qwen/Qwen2.5-Coder-0.5B (Apache-2.0), destilado do professor SlopCoder-Mongo-6.7B-v1, que é um ajuste fino do deepseek-ai/deepseek-coder-6.7b-base. Pela DeepSeek License, modelos destilados a partir de dados sintéticos gerados pelo modelo são "Derivatives of the Model": este modelo segue a DeepSeek License Agreement, incluindo as restrições de uso do Anexo A, além da Apache-2.0 do Qwen.
• Dados: 100% sintéticos (esquemas fictícios, catálogo MongoDB, documentação do Console); sem dados de clientes.
• Resultados: APT 3,57 (base: 1,46), sintaxe válida 90,7%, MongoDB válido 90,6%, intenção correta 93,6% no benchmark de 2.000 exemplos; em pedidos livres escritos à mão, 60% (72/120).
• Limitações: erros semânticos em pedidos livres, dados apenas sintéticos; revise sempre o código antes de executar.
• Versões ONNX para CPU e GPU (DirectML): esilva/SlopCoder-Mongo-0.5B-ONNX.