Instructions to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
# Run inference directly in the terminal:
llama cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
# Run inference directly in the terminal:
llama cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
# Run inference directly in the terminal:
./llama-cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX

Use Docker

docker model run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX

LM Studio
Jan
Ollama
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Ollama:
```
ollama run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
```

Unsloth Studio

How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX to start chatting

Docker Model Runner
How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Docker Model Runner:
```
docker model run hf.co/Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX
```

Lemonade

How to use Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX

Run and chat with the model

lemonade run user.DeepSeek-V4-Flash-0731-ROCmFP3-MIX-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

DeepSeek-V4-Flash-0731 — ROCmFPX mixed-precision (gu106 repair)

Mixed-precision GGUF of DeepSeek-V4-Flash-0731 for AMD Strix Halo (gfx1151) at the exact 102.32 GB deployment budget (128 GB LPDDR5X − 11.3 GB DSpark draft − headroom).

ds4-0731-gu106-v1.gguf (102,321,006,592 B) + ds4-0731-gu106-v1.gguf.gumix.bin — BOTH files are required. The .gumix.bin sidecar carries the gate/up codebooks; the loader registers 86 qtype-106 tensors from it (down codebooks are embedded in-GGUF).

Allocation

family	format	bpw
routed gate/up experts (57.7 GB)	Q2_1_ROCMFP2_MIX (106) — learned per-expert codebooks, importance-weighted, gain-corrected	2.50
routed down experts	Q3_1_ROCMFP3_MIX (105) — adaptive, ds4chat-calibrated	3.50
attention / dense / shared	Q4_0_ROCMFP4_FAST (101)	4.25
scaffolding etc.	101 / F32 passthrough	—

Every family is allocated from measured layer-output damage under a whole-file byte ceiling (geo-quant auto). An earlier revision of this repo shipped gate/up as uniform absmax (107) — a family-level GGUF bisection identified that as the cause of a 9/17→17/17 quality gap, and this artifact is the repair: same bytes, +8 questions. Q8_0 attention was also built and measured (+2.4 GB): lossless on perplexity, worthless at task level (78 vs 79/92) — this smaller artifact wins.

Evaluation (dflash serving, fused decode off, serial HC; all items held out)

suite	score
COMPSEC 17 (think budget 15488)	17/17
COMPSEC 17 (forced-close, think 2000)	16/17 answered-17
full ds4-eval 92 (COMPSEC+AIME2025+GPQA-D+SuperGPQA, think 15488)	79/92 (88/92 answered)

Calibration (c4:train importance for gate/up, ds4chat-v2 for down) excludes all 92 eval items. For comparison, the published 86.7 GB IQ2_XXS reference scores 82/92 — with 75 of the 92 items present in its own imatrix calibration set; on the genuinely held-out COMPSEC slice both score 17/17.

Serving

Requires a dflash build with qtype-105/106 decode (2026-07-28+). Set DFLASH_DS4_HC_SERIAL=1 (large-footprint stability). Keep the sidecar next to the GGUF. --ds4-fused-decode only on gfx1151 single-device; never for evals on CUDA builds.

Downloads last month: 782

GGUF

Model size

284B params

Architecture

deepseek4

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Geometric-AI/DeepSeek-V4-Flash-0731-ROCmFP3-MIX

Base model

deepseek-ai/DeepSeek-V4-Flash-0731

Quantized

(104)

this model