[FFmpeg-trac] #4421(avcodec:new): flac md5 error with LPC precision 15

[FFmpeg]

#4421: flac md5 error with LPC precision 15
------------------------------------+-----------------------------------
             Reporter:  cehoyos     |                    Owner:
                 Type:  defect      |                   Status:  new
             Priority:  normal      |                Component:  avcodec
              Version:  git-master  |               Resolution:
             Keywords:  flac        |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |
------------------------------------+-----------------------------------

Comment (by lvqcl):

 There are two functions in libavcodec/flacdsp.c: `flac_lpc_encode_c_16()`
 and `flac_lpc_encode_c_32()`. The former uses variables of type `int32_t`
 to store the sum of `coefs[j] * smp[j]`; the latter uses variables of type
 `int64_t` for this.

 Similar functions exist in libFLAC:
 `FLAC__lpc_compute_residual_from_qlp_coefficients()` and
 `FLAC__lpc_compute_residual_from_qlp_coefficients_wide()`, respectively.

 Here's how libFLAC chooses which function to use:

 {{{
 if(subframe_bps + qlp_coeff_precision + FLAC__bitmath_ilog2(order) <= 32)
         use_32bit_sum();
 else
         use_64bit_sum();
 }}}

 while FFMPEG uses the following condition:

 {{{
     if (bps > 16) {
         use_64bit_sum();
     } else {
         use_32bit_sum();
     }

 }}}

 It seems that libFLAC doesn't allow the sum to overflow while FFMPEG
 allows it, and it's a bug in FFMPEG.

 --------------------------
 The same logic applies to a choice between '''de'''coding routines, that's
 why FFMPEG is able to decode the flac file it created: a bug in the
 decoder compensates a bug in the encoder.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/4421#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker