Aggressively quantized: Q4_K_M, Q5_K_M, Q6_K, Q8_0, int4. Same model, fraction of the size.
-
dispatchAI/Qwen2.5-1.5B-Instruct-mobile-int4
Text Generation • 2B • Updated • 411 -
dispatchAI/Qwen2.5-0.5B-Instruct-mobile-int4
Text Generation • 0.6B • Updated • 670 -
dispatchAI/TinyLlama-1.1B-Chat-mobile-int4
Text Generation • 1B • Updated • 161 -
dispatchAI/Llama-3.2-1B-Instruct-Q4-mobile
Text Generation • 1B • Updated • 750