README.md

---
license: mit
---

# HPC-Quantize

## Holographic Phase Contraction for Ultra-Low-Bit LLM Quantization

**HPC-Quantize** is an experimental, MIT-licensed quantization engine for aggressively compressing large language models into extremely low-bit formats, with a particular focus on **Q2-class quantization**.

The central idea is simple:

> **At very low bitrates, quantization should be treated as a structured reconstruction problem rather than independent rounding of individual blocks.**

Instead of choosing a quantization candidate solely from its local reconstruction error, HPC generates competing reconstructions, represents them in a compact discrete state space, models interactions between neighboring blocks, and performs a global sequence optimization before writing the final GGUF.

The current production path is entirely classical. Earlier versions explored quantum-inspired state and measurement formulations; the current implementation uses a **sequential Sieve, bounded state back-action, and a 36-state Viterbi optimizer** for Q2.

---

## Why HPC?

At Q4 or Q5, a model often has enough representational freedom that many quantization strategies work reasonably well.

At Q2, the situation changes dramatically.

A block has very few representable values, so small decisions about scale, minimum, and code assignment can produce disproportionately large changes in the resulting weight tensor.

A conventional quantizer often reduces the problem to:

```text
original weights
      │
      ▼
find locally best parameters
      │
      ▼
encode quantized block
```

HPC instead treats each block as a **discrete candidate-selection problem**:

```text
original weights
      │
      ▼
generate candidate reconstructions
      │
      ▼
score candidate errors
      │
      ▼
map candidates into a compact state space
      │
      ▼
sequential Sieve
      │
      ▼
Q2 state lattice
      │
      ▼
global Viterbi optimization
      │
      ▼
select physical reconstructions
      │
      ▼
GGUF
```

This lets the optimizer preserve competing possibilities until there is enough information to make a global decision.

---

# What HPC actually does

HPC is still a **quantizer/re-quantizer**.

It does not retrain the neural network, modify the model architecture, or learn a new set of representations.

The "reconstruction" terminology refers to what happens during candidate selection: HPC explicitly constructs multiple possible low-bit approximations of the original weights and evaluates them against the source weights.

Conceptually:

$$
W \rightarrow
\{\hat W_1,\hat W_2,\ldots,\hat W_n\}
\rightarrow
\text{structured candidate selection}
\rightarrow
Q(W)
$$

The final output remains a normal quantized GGUF model.

---

# Core design

HPC combines several ideas:

* candidate reconstruction
* weighted reconstruction error
* optional importance-matrix weighting
* discrete state mapping
* sequential Sieve selection
* bounded neighboring-state back-action
* Q2 state coupling
* global Viterbi optimization
* local reconstruction-quality safeguards

The important distinction is that these components operate **together** rather than treating every quantization block as completely independent.

---

# Candidate generation

For an eligible Q2 block, HPC searches over possible quantization parameters rather than committing immediately to a single local solution.

For Q2_K, the important coupled parameters are represented conceptually as:

$$
(d,d_{\min})
$$

Candidate parameters are evaluated by reconstructing the quantized block and measuring its error against the original weights.

With an importance matrix, the error can be weighted so that sensitive dimensions contribute more heavily:

$$
E =
\sum_i w_i(x_i-\hat{x}_i)^2.
$$

This is useful because ordinary unweighted RMSE assumes every weight contributes equally to the final model behavior.

HPC can therefore evaluate:

> **How well does this candidate reconstruct the important parts of the original block?**

rather than only:

> **How small is its raw Euclidean error?**

---

# From candidates to states

Keeping every physical candidate in the global optimizer would be expensive.

HPC therefore maps candidates into a compact symbolic state representation.

The current implementation uses **six symbolic states** for each quantization parameter:

$$
d \in \{0,1,2,3,4,5\}.
$$

The symbolic states provide a compact representation of the candidate landscape.

Multiple physical candidates can belong to the same symbolic state.

This separation is important:

```text
physical candidates
        │
        ▼
symbolic state representation
        │
        ▼
global optimization
        │
        ▼
physical candidate selection
```

The symbolic state space is therefore not the same thing as the number of physical reconstructions generated during candidate search.

---

# The Q2 state space

Q2 has two coupled parameters:

$$
(d,d_{\min}).
$$

Each is represented by six symbolic states.

Therefore the joint Q2 space contains:

$$
6\times6=36
$$

states.

The state index is:

$$
s=6q_D+q_M
$$

where:

$$
q_D,q_M\in\{0,\ldots,5\}.
$$

The resulting lattice is:

```text
          dmin
          0   1   2   3   4   5
        ┌────────────────────────
      0 │  0   1   2   3   4   5
      1 │  6   7   8   9  10  11
d     2 │ 12  13  14  15  16  17
      3 │ 18  19  20  21  22  23
      4 │ 24  25  26  27  28  29
      5 │ 30  31  32  33  34  35
```

The 36-state representation is a **global optimization space**, not a claim that only 36 physical quantization candidates exist.

---

# Candidate probabilities

Rather than treating each candidate as simply "best" or "not best", HPC can preserve information about the relative quality of competing candidates.

Candidate errors are converted into probability-like weights using a Boltzmann-style transformation:

$$
P_i \propto e^{-T(E_i-E_{\min})}.
$$

This creates a soft candidate distribution.

The practical consequence is important:

> A candidate that is slightly worse than the local minimum can remain relevant if it belongs to a useful region of the state space.

That makes the search less eager to collapse immediately to a single local minimum.

---

# The Sieve

The current production architecture uses a **sequential Sieve**.

Rather than treating blocks as fully independent, neighboring state distributions influence one another.

For a candidate state \(d\), the implementation applies a bounded compatibility penalty based on neighboring probability:

$$
B_j(d)=1-0.15p_j(d).
$$

The combined Sieve score can be written conceptually as:

$$
S(d)=p_0(d)\prod_j(1-0.15p_j(d)).
$$

The effect is deliberately modest.

If a neighboring site is strongly concentrated on the same state, that state becomes somewhat less attractive locally.

The Sieve therefore encourages **state diversity and compatibility** without forcing an alternating pattern.

---

# Sieve slack

The Sieve also avoids immediately discarding every candidate that is not the top local state.

A bounded slack region allows near-optimal states to survive the first selection stage.

Conceptually:

```text
best state
    │
    ├── keep
    │
    ├── keep near-optimal alternatives
    │
    └── discard clearly inferior states
```

This prevents the candidate distribution from collapsing too early.

---

# Sequential conditioning

Once the current state is selected, its influence is propagated into neighboring sites.

The selected state therefore affects subsequent decisions.

This gives the process a sequential character:

$$
P(d_1)
\rightarrow
P(d_2|d_1)
\rightarrow
P(d_3|d_1,d_2)
\rightarrow \cdots
$$

The quantization process is consequently no longer just a collection of independent block decisions.

---

# Viterbi optimization

After the Sieve, Q2 state probabilities are expanded into the full **36-state joint lattice**.

Each block receives a local cost combining reconstruction error with state probability.

Conceptually:

$$
C_i(s)
=
E_i(s)-\lambda \log P_i(s).
$$

HPC then adds a transition cost between neighboring blocks.

A simple form is:

$$
T(s',s)
\propto
|q_D-q'_D|+
|q_M-q'_M|.
$$

This is Manhattan distance on the 6×6 state lattice.

The dynamic-programming recurrence is:

$$
DP_i(s)
=
C_i(s)
+
\min_{s'}
\left[
DP_{i-1}(s')+T(s',s)
\right].
$$

The result is a **globally optimized sequence of Q2 states**.

This is one of the major differences between HPC and purely local quantization.

---

# Local safety

Global regularization should not be allowed to produce obviously poor local reconstructions.

After global selection, HPC can compare the chosen state against the locally best reconstruction.

A sufficiently large local improvement can trigger a local override.

This gives the optimizer a safety mechanism:

```text
global structure
      │
      ▼
candidate selected
      │
      ▼
is the local reconstruction much better?
      │
   ┌──┴──┐
   │     │
  yes    no
   │     │
local   keep
winner  global
        winner
```

The goal is to prevent the global objective from becoming disconnected from actual reconstruction quality.

---

# Error geometry

HPC can also use structured error decomposition rather than treating every error component identically.

One experimental component uses a D₆/Vesica-style decomposition of paired error terms.

For a pair of error components:

$$
v=e_p+e_{p+h}
$$

$$
w=e_p-e_{p+h}.
$$

This separates the error into different modes before applying the final weighting.

The intent is to distinguish error geometry rather than assuming that all directions in weight space have identical consequences.

This is an experimental feature of the HPC objective rather than a requirement of GGUF or Q2_K itself.

---

# Building

HPC-Quantize is intended to be used alongside a GGUF/llama.cpp workflow.

Typical dependencies include:

```bash
sudo apt install \
    gcc \
    libgmp-dev \
    libmpfr-dev \
    python3 \
    python3-numpy
```

Build the native quantization component:

```bash
make -f makefile.quantize
```

The resulting library/binary names may vary with the current revision.

---

# Mixed-precision workflows

HPC is primarily intended to solve the problem of **aggressive compression**, not to force every tensor in a model into identical precision.

A practical deployment may therefore retain higher precision for particularly sensitive tensors and use Q2 for the bulk of the model.

For example:

```text
Model
 ├── embeddings        → higher precision
 ├── normalization     → preserved
 ├── attention         → Q4 / promoted
 ├── FFN / experts     → Q2
 └── other large mats  → Q2
```

The optimal allocation is model-dependent.

---

# Why Q2?

Q2 is where conventional quantization becomes particularly unforgiving.

At higher precision, the quantizer has many representational degrees of freedom.

At Q2, many distinct original weight values must share a very small set of reconstruction values.

This means:

$$
\text{small parameter change}
\rightarrow
\text{large discrete reconstruction change}.
$$

HPC is designed around this regime.

Rather than assuming the locally nearest reconstruction is always globally best, it explicitly searches among competing discrete configurations.

---

# HPC versus conventional quantization

A simplified conventional pipeline is:

```text
weight block
    │
    ▼
estimate scale/minimum
    │
    ▼
round values
    │
    ▼
write block
```

A simplified HPC pipeline is:

```text
weight block
    │
    ▼
generate competing reconstructions
    │
    ▼
score candidates
    │
    ▼
map to symbolic states
    │
    ▼
Sieve + neighboring interaction
    │
    ▼
construct Q2 state lattice
    │
    ▼
Viterbi global optimization
    │
    ▼
select physical reconstructions
    │
    ▼
write Q2_K
```

The difference is not that HPC stops being quantization.

The difference is **how much structure it retains before committing to the final quantized representation**.

---

# A useful way to think about HPC

HPC can be viewed as three nested optimization problems:

### Local reconstruction

> Which low-bit approximation best represents this block?

### State inference

> Which region of the discrete candidate space is promising?

### Global sequence optimization

> Which sequence of candidate states produces the best overall configuration?

That can be summarized as:

$$
\boxed{
\text{reconstruction}
+
\text{state inference}
+
\text{global optimization}
}
$$

rather than:

$$
\boxed{
\text{independent rounding}
}
$$

---

# Experimental nature

HPC is research software.

It should not be assumed that:

* lower RMSE always produces better model behavior;
* lower perplexity always produces better reasoning;
* one quantization strategy wins on every architecture;
* Q2 quality transfers perfectly between models;
* state-interaction parameters are universally optimal.

The correct way to evaluate HPC is with a combination of:

* reconstruction error
* perplexity
* reasoning benchmarks
* mathematical evaluation
* coding tasks
* long-context tests
* instruction following
* qualitative generation
* memory usage
* inference speed

The objective of HPC is not to optimize one number in isolation.

---

# Current architecture

The project originally explored a more explicitly quantum-inspired formulation involving state amplitudes, phase operations, graph coupling, Fourier/IDFT transforms, and sequential measurement.

The current engine has moved toward a more explicit classical formulation:

```text
Historical approach
───────────────────
candidate error
      ↓
amplitudes / phase
      ↓
graph coupling
      ↓
measurement
      ↓
back-action


Current approach
────────────────
candidate error
      ↓
probability distribution
      ↓
sequential Sieve
      ↓
bounded back-action
      ↓
36-state Q2 lattice
      ↓
Viterbi
```

The mathematical intuition of interacting discrete states remains, but the current production implementation is classical and deterministic.

---

# Future directions

The state lattice is intentionally compact.

The 36-state Q2 space is:

$$
6\times6=36.
$$

That does **not** mean the physical candidate space must contain only 36 candidates.

Possible future work includes:

* more symbolic states per parameter;
* multiple physical candidates retained per symbolic state;
* beam search inside individual states;
* hierarchical state refinement;
* adaptive state resolution;
* larger candidate beams for difficult tensors;
* tensor-dependent state cardinality;
* improved transition models;
* architecture-specific state priors.

One particularly interesting extension is to retain multiple physical candidates for each symbolic state:

$$
36\times K.
$$

This would preserve more of the physical reconstruction landscape while keeping the coarse 6×6 state structure.

---

# What HPC is trying to preserve

At ultra-low precision, numerical error is inevitable.

The goal is therefore not:

> **make every weight numerically perfect.**

The goal is:

> **spend the available representational capacity where it matters most, preserve competitive reconstruction alternatives long enough for global selection, and avoid treating every block as an isolated rounding problem.**

That is the central design philosophy of HPC-Quantize.

---

# License

HPC-Quantize is released under the **MIT License**.

The licensing terms of any model quantized with HPC remain separate from the HPC software license.

Always verify the license of the underlying base model before redistribution.

---

# Status

**Experimental / research software**

The current engine is actively evolving, particularly around ultra-low-bit Q2 quantization.

The project currently prioritizes:

* low-bit reconstruction quality
* model coherence
* reasoning preservation
* structured state selection
* aggressive memory reduction

over compatibility with any single traditional quantization metric.

---

## In one sentence

**HPC-Quantize is a structured ultra-low-bit quantizer that searches over competing Q2 reconstructions, reasons about them as interacting discrete states, and uses global sequence optimization to choose the final GGUF configuration.**

---

## Acknowledgements

HPC-Quantize builds on the broader GGUF and llama.cpp ecosystem and is intended to interoperate with existing llama.cpp-based tooling.

The project also explores ideas inspired by discrete graphical models, sequential inference, information-weighted reconstruction, and quantum-inspired state representations.