quantize
To approximate a continuously varying signal by one whose amplitude can only have a set of discrete values
quantize: turn infinite into finite
To quantize a signal is to map its continuous range of values onto a discrete set of fixed levels. An analog-to-digital converter does this: it samples the incoming voltage at regular intervals and rounds each sample to the nearest of a finite number of possible output values. If your converter has 8-bit resolution, it divides the input range into 256 equally spaced steps; a 16-bit converter gives 65,536 steps. The rounding error is called quantization error or quantization noise, and it is unavoidable.
The number of bits in the converter determines resolution. A 12-bit ADC quantizing a 0 to 10 volt signal creates steps of roughly 2.4 millivolts. A 24-bit converter on the same range produces steps below 1 microvolt. More bits mean finer granularity but also higher cost and slower conversion speed. The choice depends on the signal's actual content: quantizing a 60 Hz power waveform to 12 bits is usually enough; precision instrumentation or audio production often demands 16 or 24 bits.
Quantization error and signal loss
When a signal lies between two quantization levels, the converter must round it up or down. This introduces error equal to plus or minus half a least significant bit (LSB). In a measurement system, quantization error combines with other sources like sensor noise and temperature drift. If your signal is weak relative to the step size, quantization noise becomes audible in audio or visible as banding in images.
Dithering partially solves this problem. By adding a small amount of random noise before quantization, you spread the rounding error across a range of values rather than clustering it at fixed levels. Paradoxically, noise improves perceived smoothness. This technique is standard in digital audio and image processing.
The term quantize comes from physics, where it refers to the discrete energy levels in atoms. Engineers borrowed it because the mathematical problem is identical: a continuous variable forced into discrete bins. You encounter quantization not only in ADCs but also in digital signal processors, image compression algorithms, and neural network weights being reduced to lower precision for faster inference.