Patent attributes
A computational device includes an input memory, which receives a first array of input numbers having a first precision represented by N bits. An output memory stores a second array of output numbers having a second precision represented by M bits, M<N. Quantization logic reads the input numbers from the input memory, extracts from each input number a set of M bits, at a bit offset within the input number that is indicated by a quantization factor, and writes a corresponding output number based on the extracted set of bits to the second array in the output memory. A quantization controller sets the quantization factor so as to optimally fit an available range of the output numbers in the second array to an actual range of the input numbers in the first array in extraction of the M bits from the input numbers.
