Unified manuscript

Distribution-Controlled Selective Quantization

J. R. Landers, May 2026

Distortion, allocation, and margin-survival accuracy curves for a distribution-shaping view of mixed-precision quantization.

PDF manuscript Selective quantization Margin survival Distribution control ReLU classifier experiment
Abstract. Worst-case quantization bounds price every coordinate as if every perturbation mattered equally. A distribution-controlled view does something sharper: it spends low precision where the model is insensitive on the data it actually sees. For linear readouts, blockwise ReLU networks, and coefficient-feature classifiers, the same first-order product governs distortion, \[ \text{selection probability}\times \text{error scale}\times \text{sensitivity}. \] This unified manuscript joins the selective-quantization note with its margin-survival extension. Margins convert cumulative output distortion into training-accuracy curves: sorted least-sensitive-first quantization accumulates distortion slowly, and accuracy survives until that cumulative distortion reaches the body of the margin distribution. On handwritten digits, a closed-form lognormal-plus-Weibull margin-survival model fits the least-sensitive-first coefficient-feature sweep with an RMSE of \(0.159\) percentage points. A separate ReLU bit-allocation experiment reaches full-precision test accuracy at \(4.01\) average bits and exceeds uniform 4-bit accuracy at \(3.43\) average bits.

1. The Question

Quantization buys cheaper storage and arithmetic by replacing high-precision parameters with lower-precision approximations. The blunt analysis treats the resulting error as if it can hit every coordinate at once with equal consequence. That is safe, but it misses the geometry of a trained model: some directions move the output a great deal, while others barely register on the data distribution.

Selective quantization asks not simply "how large is the rounding error?" but "where does that error land?" The right object is anisotropic: low precision should be allocated heavily to directions with small sensitivity, lightly to directions with large sensitivity, and smoothly or abruptly depending on the objective used to choose the allocation.

\[ \text{quantization schedule} \longrightarrow \text{cumulative distortion} \longrightarrow \text{margin survival} \longrightarrow \text{accuracy curve}. \]

Start with a linear readout \(f_w(x)=w^\top x+b\). Quantization perturbs each coordinate only when a selector \(M_i\) is active:

\[ \widetilde w_i=w_i+M_i\xi_i,\qquad M_i\sim \operatorname{Bernoulli}(q_i),\qquad |\xi_i|\le \varepsilon_i . \]

The data distribution enters through a sensitivity score such as \(s_i=\mathbb E_{\mathcal D}|X_i|\). The basic selective family is

\[ q_i \propto (s_i+\lambda)^{-\alpha},\qquad \alpha>0,\ \lambda>0, \]

so low-sensitivity directions receive more of the quantization mass. This changes the question from "how large is the rounding error?" to "where does the rounding error land?"

Distribution

Sensitivity scores measure how strongly the model uses each coordinate on the data it actually sees.

Allocation

Quantization probabilities and bit widths determine where low precision is applied.

Distortion

The relevant first-order term is the aligned product \(q_i\varepsilon_i s_i\), not the error scale alone.

2. Distribution-Controlled Gates

Let \(z=\log s\), and let \(\rho(z)\) be the population density of log-sensitivities. Applying a selective rule produces the quantization density

\[ \pi(z)=\frac{\rho(z)q(z)}{\int \rho(u)q(u)\,du}. \]

For a lognormal population and \(q(s)\propto s^{-\alpha}\), the quantization mass shifts left by \(\alpha\sigma^2\) in log-sensitivity. The high-sensitivity tail is still present in the model, but it receives very little of the low-precision budget.

\[ z\mid\hbox{quantized} \sim \mathcal N(m-\alpha\sigma^2,\sigma^2). \]
Population sensitivity density and shifted quantization mass
A lognormal sensitivity population reweighted by a selective rule. Quantization mass moves away from the top sensitivity tail.
Hard threshold, logistic gates, and cumulative distortion curves
Hard thresholding is the zero-temperature face of a smoother logistic gate family. The cumulative distortion curve tracks the same allocation geometry.

3. First-Order Distortion

Proposition 1. Selective quantization bound

For every fixed input \(x\in\mathbb R^d\),

\[ \mathbb E_Q\!\left[|f_{\widetilde w}(x)-f_w(x)|\right] \le \sum_{i=1}^d q_i\varepsilon_i |x_i|+\varepsilon_b. \]

Averaging over \(X\sim\mathcal D\) gives

\[ \mathbb E_{X,Q}\!\left[|f_{\widetilde w}(X)-f_w(X)|\right] \le \sum_{i=1}^d q_i\varepsilon_i s_i+\varepsilon_b. \]

Uniform quantization sets \(q_i=q\) and \(\varepsilon_i=\varepsilon\), giving the coarser \(q\varepsilon\sum_i s_i+\varepsilon_b\). Selective quantization improves the same bound by moving mass to coordinates where the product is cheap.

Proposition 2. Budget-optimal selection

With common error scale \(\varepsilon_i=\varepsilon\) and budget \(\sum_iq_i=K\), minimizing \(\varepsilon\sum_i q_is_i\) places mass on the \(K\) least sensitive coordinates. If \(K\) is not an integer, one coordinate is fractional.

Smoothing the threshold

Abrupt rules are sometimes undesirable. They are hard to tune, brittle under noisy sensitivity estimates, and inconvenient when the selection mechanism is trained by gradient methods. Adding a convex entropy penalty softens the vertex solution:

\[ \min_{q\in[0,1]^d} \sum_i q_i c_i +\mu\sum_i\left[q_i\log q_i+(1-q_i)\log(1-q_i)\right] \quad \text{s.t.}\quad \sum_i q_i=K . \]

Proposition 3. Smooth allocation

With \(c_i=\varepsilon_i s_i\) and \(\mu>0\), the unique minimizer is logistic in the coordinate cost:

\[ q_i^\star = \frac{1}{1+\exp((c_i+\nu)/\mu)}, \]

where \(\nu\) is chosen so that \(\sum_i q_i^\star=K\). As \(\mu\to0\), the solution steepens into the threshold rule.

When signs matter

If quantization errors are independent, bounded, and mean zero, signed terms cancel and concentration gives a sharper tail. Deterministic rounding is different: the fixed signs make the bias term capable of cancellation, while the gate variance contains \(q_i(1-q_i)\), which is concave and vanishes at \(q_i=0\) and \(q_i=1\).

Proposition 4. Selection variance and tail

Let \(u_i=\beta_i x_i\) and \(Z(x)=\sum_iM_i u_i\), with \(M_i\sim\operatorname{Bernoulli}(q_i)\). Then

\[ \mathbb E Z(x)=\sum_iq_i u_i,\qquad \operatorname{Var}(Z(x))=\sum_i q_i(1-q_i)u_i^2 . \]

At deterministic vertices \(q_i\in\{0,1\}\), the gate variance disappears and the residual problem becomes choosing a subset whose signed rounding errors cancel.

The same argument lifts to ReLU networks by replacing coordinates with parameter blocks. If \(G_j\) is a local block sensitivity, then inside a visited activation region,

\[ \mathbb E_Q\!\left[\|f_{\widetilde\theta}(x)-f_\theta(x)\|\right] \lesssim \sum_j q_j\varepsilon_jG_j . \]
Curvature weighted distortion for signed rounding choices
Curvature decides whether the allocation hedges or commits. Entropy smoothing favors interior gates; deterministic-rounding variance vanishes at the corners, turning the remaining problem into signed subset selection.

4. Distribution Shaping as Constrained Optimization

The allocation rule can be written as an explicit constrained problem. Given a target distortion \(\tau\), choose probabilities and bit widths to minimize memory:

\[ \min_{\phi}\; \mathcal M(\phi) \quad\text{subject to}\quad \mathcal D(\phi)= \int \rho(z)q_{\phi}(z)\varepsilon(b_{\phi}(z))e^z\,dz \le \tau . \]

A regularizer can shape the induced quantization distribution \(\pi_\phi(z)\), for example toward a uniform target, a lognormal target, or a mixture between them:

\[ \mathcal J(\phi,\lambda) = \mathcal M(\phi) +\lambda[\mathcal D(\phi)-\tau]_+^2 +\eta\,\mathcal R(\phi). \]

Proposition 5. Continuous-bit water filling

For distortion terms \(a_j2^{-b_j}\), memory weights \(m_jb_j\), and box constraints \(b_{\min}\le b_j\le b_{\max}\), the relaxed optimizer is

\[ b_j^\star = \operatorname{clip}_{[b_{\min},b_{\max}]} \log_2\!\left(\frac{\lambda a_j\log 2}{m_j}\right). \]

More sensitive or curvature-heavy blocks receive more bits; cheap blocks fall to the lower precision floor.

5. From Distortion to Margins

The preceding theory controls score movement. A classifier's accuracy is governed by whether that movement crosses decision margins. For a multiclass classifier with scores \(f_c(x)\), define the clean training margin of example \((x_i,y_i)\) by

\[ m_i=f_{y_i}(x_i)-\max_{c\ne y_i} f_c(x_i). \]

The example is correctly classified when \(m_i>0\). After quantizing a cumulative set \(S_k\), write the score perturbation as \(\delta_c^{(k)}(x_i)=f_c^{(k)}(x_i)-f_c(x_i)\). The quantized margin obeys

\[ m_i(k) \ge m_i - \left|\delta_{y_i}^{(k)}(x_i)\right| - \max_{c\ne y_i}\left|\delta_c^{(k)}(x_i)\right|. \]

For a linear score model, quantizing coefficient feature \(j\) changes class \(c\)'s score by \(\delta_c^{(k)}(x_i)=\sum_{j\in S_k}\Delta W_{cj}x_{ij}\). A schedule-level proxy replaces the sample-specific perturbation by cumulative first-order exposure:

\[ D(k)=\sum_{j\in S_k} c_j,\qquad c_j=\varepsilon_jG_j. \]

Proposition 6. Accuracy as margin survival

Let \(S_k\) be a cumulative quantization schedule and let \(D(k)=\sum_{j\in S_k}c_j\). If the induced margin perturbation is approximated by a common scale factor times cumulative distortion, \(|\Delta m_i(k)|\approx\gamma D(k)\), then

\[ A(k)\approx A_0\left[1-F_M(\gamma D(k))\right], \]

where \(F_M\) is the empirical CDF of clean positive margins among initially correct training examples and \(A_0\) is full-precision training accuracy. With endpoint normalization to the fully quantized accuracy \(A_\infty=A(d)\),

\[ A(k) \approx A_0 - (A_0-A_\infty) \frac{F_M(\gamma D(k))-F_M(0)} {F_M(\gamma D(d))-F_M(0)}. \]

\[ \text{accuracy curve} \approx \text{margin survival} \circ \text{cumulative quantization distortion}. \] The selective-quantization product supplies the distortion path \(D(k)\). The margin distribution turns that path into discrete classification errors.

6. Closed-Form Least-Sensitive-First Accuracy

Specialize to the least-sensitive-first schedule. Let \(\phi=k/d\) be the fraction of quantized coordinates and suppose coordinate costs are approximately lognormal:

\[ C=\varepsilon G,\qquad \log C\sim\mathcal N(\mu,\sigma_s^2). \]

The least-sensitive-first cumulative distortion fraction is

\[ R(\phi) = \frac{\mathbb E\left[C\,\mathbf 1\{C\le Q_\phi\}\right]}{\mathbb E C} = \Phi\!\left(\Phi^{-1}(\phi)-\sigma_s\right). \]

This is the plateau generator: a large fraction of coordinates can be quantized while accumulating only a small fraction of the total sensitivity-weighted distortion. Approximate the empirical margin CDF by a Weibull-type failure law,

\[ F_M(t)\approx 1-\exp\left[-\left(\frac{t}{\tau}\right)^\beta\right]. \]

Absorbing the margin scale into \(\tau\), composition gives the explicit least-sensitive-first accuracy curve

\[ A_{\rm blue}(\phi) = A_0 - (A_0-A_\infty) \frac{1-\exp\left[-(R(\phi)/\tau)^\beta\right]} {1-\exp\left[-(1/\tau)^\beta\right]}. \]
Cumulative coefficient-feature quantization accuracy curves with a margin-survival fit
Cumulative coefficient-feature quantization on UCI/scikit-learn handwritten digits. The 64 pixel coordinates are expanded to 2,144 polynomial design features and a multiclass Ridge classifier is quantized cumulatively by coefficient-feature column using deterministic 2-bit signed mid-tread quantization. Shaded bands show plus or minus one standard deviation across 40 stratified training splits. The dashed black curve is the margin-survival fit.
Quantity Symbol Fitted value
Full-precision training accuracy \(A_0\) 99.923%
Fully quantized training accuracy \(A_\infty\) 32.065%
Log-cost spread \(\sigma_s\) 0.651
Margin scale \(\tau\) 0.905
Margin-shape exponent \(\beta\) 3.214
Fit RMSE - 0.159 percentage points

The fit is a structural check rather than a benchmark claim. The original formalism predicts that least-sensitive-first quantization accumulates distortion slowly. The margin formalism predicts that accuracy remains stable until cumulative distortion enters the body of the margin distribution. Together they explain the plateau-then-collapse shape.

Reproduction files: summary CSV, raw split CSV, and source figure PNG.

7. ReLU Net Experiment

The paper includes a small post-training quantization test on the scikit-learn digits dataset. A ReLU classifier with architecture \(64\to 96\to 10\) is trained in full precision. Parameter sensitivity is estimated by

\[ s_i=|\theta_i|\sqrt{\widehat F_i}+10^{-12}, \]

where \(\widehat F_i\) is an empirical diagonal Fisher estimate on a calibration split. The optimizer then chooses integer bits \(b_i\in\{2,\ldots,8\}\) subject to the same proxy distortion budget used by a uniform baseline.

Digits MLP quantization experiment showing accuracy versus average bits and bit allocation by sensitivity decile
The optimized allocation reaches full-precision test accuracy of 97.8 percent at 4.01 average bits. Uniform 4-bit quantization reaches 95.3 percent. At 3.43 average bits, the optimized policy reaches 95.6 percent, slightly above uniform 4-bit at lower memory.
Policy Average bits Compression vs. fp32 Test accuracy Interpretation
Full precision 32.00 1.00x 97.8% Reference model.
Uniform 4-bit 4.00 8.00x 95.3% Same average memory as the strongest selective point, but lower accuracy.
Selective, target 6 4.01 7.98x 97.8% Matches full precision at roughly one eighth the fp32 memory.
Selective, target 5 3.43 9.34x 95.6% Beats uniform 4-bit accuracy with less memory.

Reproduction files: results CSV, bit distribution bins, and experiment summary.

8. One Product, Five Geometries

\[ \sum_j q_j\,\varepsilon_j\,G_j \qquad \text{selection probability} \times \text{error scale} \times \text{sensitivity}. \] This product is the invariant. The allocation rule and the observed accuracy curve are determined by the curvature, constraints, and margin distribution built around it.

Regime Shape of objective or readout Behavior
First-order magnitude Linear in \(q_i\) Vertex solution; quantize least-sensitive coordinates first.
Entropy-relaxed selection Convex smoothing term Interior logistic gates; threshold recovered as \(\mu\to0\).
Deterministic rounding Concave fluctuation \(q_i(1-q_i)\) plus signed bias Commit to corners, then choose subsets whose signed errors cancel.
Distribution control Memory with loss constraint Shape the quantization distribution along the Pareto frontier.
Margin survival Thresholding by clean classification margins Accuracy remains stable until cumulative distortion reaches the margin distribution.

The practical message is stable across the variants: spend numerical imprecision where sensitivity is small. What changes is the mode of spending. Linear costs commit by a threshold, convex relaxations hedge smoothly, deterministic signed errors return the problem to combinatorial commitment, constrained optimization turns allocation into direct distribution control, and margins turn cumulative distortion into the observed accuracy curve.

References

  1. Y. LeCun, J. S. Denker, and S. A. Solla. Optimal Brain Damage. Advances in Neural Information Processing Systems, 1989.
  2. Z. Dong, Z. Yao, Y. Cai, D. Arfeen, A. Gholami, M. W. Mahoney, and K. Keutzer. HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks. Advances in Neural Information Processing Systems, 2020.
  3. E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. International Conference on Learning Representations, 2023.
  4. S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan. Deep Learning with Limited Numerical Precision. International Conference on Machine Learning, 2015.
  5. C. Louizos, M. Welling, and D. P. Kingma. Learning Sparse Neural Networks through \(L_0\) Regularization. International Conference on Learning Representations, 2018.
  6. E. Jang, S. Gu, and B. Poole. Categorical Reparameterization with Gumbel-Softmax. International Conference on Learning Representations, 2017.
  7. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, 2nd edition, 2006.
  8. F. Pedregosa et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 2011.