GLM-5.2-VL

GLM-5.2 that can see. This model grafts a vision encoder onto the GLM-5.2 MoE language model through a lightweight multimodal bridge, packaged as a single out-of-the-box transformers model. The GLM-5.2 language model weights are unmodified — its text, code and reasoning abilities are exactly those of the original model — with the routed experts stored in bitsandbytes NF4 (4-bit) to cut the footprint from 1.4 TB to ~417 GB.

Note: GLM-5.2-VL is a community vision graft on top of GLM-5.2. It is not an official Zhipu AI / Z.ai release.

• Vision encoder: Gemma-4 vision tower (frozen, fp32), up to 1120 visual tokens per image with native aspect-ratio-preserving resolution

• LLM: GLM-5.2 (78-layer MoE, 256 routed experts, DSA sparse attention), weights identical to the original release

• Quantization: routed experts NF4 (blocksize 64); attention, routers, shared experts, embeddings in bf16; vision stack in fp32. Dequantization happens per-hit-expert at runtime and computes in standard bf16 matmuls — no custom GEMM kernels

• Thinking: inherits GLM-5.2's <think>...</think> reasoning format and chat template

  Evaluation

MMMU validation: 52.0% (468/900, zero-shot, greedy decoding, thinking enabled, all 30 subjects, no MMMU data anywhere near training).

MMMU discipline group	correct	accuracy
Humanities & Social Science	85/120	70.8%
Art & Design	67/120	55.8%
Health & Medicine	79/150	52.7%
Science	75/150	50.0%
Tech & Engineering	97/210	46.2%
Business	65/150	43.3%
Overall	468/900	52.0%

benchmark	score
text / code benchmarks	identical to GLM-5.2 (LM weights untouched)

Quickstart

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "YOUR_ORG/GLM-5.2-VL"
processor = AutoProcessor.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    repo, trust_remote_code=True, device_map="auto",
).eval()

image = Image.open("photo.jpg").convert("RGB")
batch = processor(images=image, text="Describe this image in detail.")
batch = {k: v.to(model.device) for k, v in batch.items()}

with torch.no_grad():
    out = model.generate(**batch, max_new_tokens=1024, do_sample=False)

text = processor.decode(out[0][batch["input_ids"].shape[1]:], skip_special_tokens=True)
print(text.split("</think>")[-1].strip())   # strip the thinking block

Or run the bundled script:

python example_inference.py --image test_images/chart.png \
    --prompt "What is the highest value in this chart?"

Multi-turn / multi-image via messages (images are attached in order):

batch = processor(
    images=[img1, img2],
    messages=[{"role": "user", "content": "Compare these two charts."}],
)

Thinking mode

By default the processor renders prompts with thinking disabled — this is the most stable and fastest format for perception tasks (description, OCR, classification). For hard visual reasoning you can enable GLM's <think>...</think> mode:

batch = processor(images=image, text=question, enable_thinking=True)
out = model.generate(**batch, max_new_tokens=4096)   # thinking needs budget

With thinking enabled, always give a generous max_new_tokens and parse the answer after </think>; think-mode output on visual inputs can be verbose.

Hardware

setup	works?
2 × 141 GB+ (e.g. B300/H200-class)	✅ minimum
6-8 × 80 GB (A100/H100)	✅ comfortable, device_map="auto"
CPU offload	technically possible, slow

~417 GB of weights + KV cache. bitsandbytes must be installed (pip install bitsandbytes); no other custom dependencies.

Evaluation

benchmark	score
MMMU (validation, 900 questions, zero-shot)	52.0%
text / code benchmarks	identical to GLM-5.2 (LM weights untouched)

The multimodal bridge is quantization-aware: it was tuned against the NF4 weights shipped here, so this 4-bit package is the reference deployment, not a degraded variant.

Because the language model is byte-identical to stock GLM-5.2 (modulo NF4 storage of the experts), text-only capabilities are preserved by construction.

Prompt format

Images are represented in the token stream as <|begin_of_image|> + n × <|image|> + <|end_of_image|> (n ≤ 1120, depends on aspect ratio). The processor handles this automatically — just pass PIL images. GLM-5.2's standard chat template is used unchanged.

Limitations

• Strongest on natural photos, documents, charts, screenshots and OCR. Heavily stylized content (anime, illustrations, abstract art) is out-of-distribution and descriptions may be unreliable.
• Visual grounding is answer-oriented; precise localization (boxes/points) is not supported.
• NF4 expert storage trades a small amount of quality for a 3.4× size reduction. For maximum fidelity, serve the bf16 variant.
• Inherits the biases and failure modes of its component models.

License

Mixed: the GLM-5.2 language model is MIT; the vision encoder derives from Gemma-4 and remains subject to its original terms of use. You are responsible for compliance with both.