Spaces:
Running on Zero
Running on Zero
Finalize public CUDA control documentation
Browse files- README.md +14 -6
- __pycache__/app.cpython-313.pyc +0 -0
README.md
CHANGED
|
@@ -13,14 +13,22 @@ license: apache-2.0
|
|
| 13 |
# Embedding Quantization CUDA Control
|
| 14 |
|
| 15 |
A bounded ZeroGPU control lane for the matched local MLX embedding
|
| 16 |
-
quantization experiment.
|
| 17 |
-
|
| 18 |
-
|
| 19 |
|
| 20 |
The second milestone adds CUDA-native bitsandbytes INT8 and NF4 controls.
|
| 21 |
They are deliberately reported as separate quantizers and are not treated as
|
| 22 |
equivalents of MLX Q, oQ, or oQe formats.
|
| 23 |
|
| 24 |
-
Matched BF16 controls are
|
| 25 |
-
Qwen3-Embedding-
|
| 26 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
# Embedding Quantization CUDA Control
|
| 14 |
|
| 15 |
A bounded ZeroGPU control lane for the matched local MLX embedding
|
| 16 |
+
quantization experiment. It reproduces the frozen Qwen3-Embedding-0.6B BF16
|
| 17 |
+
vectors on CUDA and compares them with the saved MLX BF16 vectors. CUDA results
|
| 18 |
+
do not reproduce MLX/Metal performance.
|
| 19 |
|
| 20 |
The second milestone adds CUDA-native bitsandbytes INT8 and NF4 controls.
|
| 21 |
They are deliberately reported as separate quantizers and are not treated as
|
| 22 |
equivalents of MLX Q, oQ, or oQe formats.
|
| 23 |
|
| 24 |
+
Matched BF16, INT8, and NF4 controls are available for
|
| 25 |
+
Qwen3-Embedding-0.6B, GTE-Qwen2-1.5B, and Qwen3-Embedding-8B. Quantized CUDA
|
| 26 |
+
vectors are evaluated against their same-loading-path CUDA BF16 reference.
|
| 27 |
+
|
| 28 |
+
GTE-Qwen2-1.5B failed the loading-path and MLX/CUDA parity checks. Its CUDA
|
| 29 |
+
quantized results remain valid only as same-path comparisons against the direct
|
| 30 |
+
CUDA BF16 reference and must not be mixed into cross-runtime vector trends.
|
| 31 |
+
|
| 32 |
+
Frozen inputs, saved vectors, raw metrics, locked model revisions, and the
|
| 33 |
+
portable client are published separately in
|
| 34 |
+
[`TiGa-RCE/embedding-quant-repro-2026-07-28`](https://huggingface.co/datasets/TiGa-RCE/embedding-quant-repro-2026-07-28).
|
__pycache__/app.cpython-313.pyc
CHANGED
|
Binary files a/__pycache__/app.cpython-313.pyc and b/__pycache__/app.cpython-313.pyc differ
|
|
|