--iq2auto

Upload 4 files

IQ2_XS

optimizations for normal llama

Upload 2 files

Requires custom llama to use with this quant method

Fold test

Fold prototype

Prototype: Produces NAN values, will not work in llama-server, do not download

Fold test

Fix OOM on large tensors

Fold interleave

DC cancellation buff

Prototype build

I prefer Q2 honestly.. but leaving IQ1 staging in for posterity

IQ1

Super small quants

Lower RMSE, improve error shaping

--analog-imatrix: Reduce error in output by 15%

Replace Shor's with accelerated sieve

Q8_0 tied embeddings

Upload 5 files

Upload 3 files

Qwen attention tensors

Update code comments

Tensor tweak

Experimental support for other LLMs

It's only calibrated for Gemma, atm.