In order to represent arbitrary signals by a finite number of bits, these signals are quantized using an appropriate quantization rule. Usually, these quantizers are designed with respect to the continuous signal and its distribution and the rate-distortion theory provides the required number of quantization bits in order to realize a pre-defined distortion measure between the analog signal and its digital representation. In contrast, with the information bottleneck method (IBM) quantizers are designed to maximize the mutual information between the quantizer output and a source signal of interest.

Details