ISTA-DASLab
/

c4ai-command-r-plus-AQLM-2Bit-1x16

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions

Official AQLM quantization of CohereForAI/c4ai-command-r-plus .

For this quantization, we used 1 codebook of 16 bits.

Results:

Model	Quantization	MMLU (5-shot)	Model size, Gb
CohereForAI/c4ai-command-r-v01	None	0.7425	208
	1x16	0.6795	31.9

Downloads last month: 3

Safetensors

Model size

16B params

Tensor type

F16

·

I16

·

Collection including ISTA-DASLab/c4ai-command-r-plus-AQLM-2Bit-1x16

AQLM

AQLM quantized LLMs • 21 items • Updated 15 days ago • 46

Paper for ISTA-DASLab/c4ai-command-r-plus-AQLM-2Bit-1x16

Extreme Compression of Large Language Models via Additive Quantization

Paper • 2401.06118 • Published Jan 11, 2024 • 14