July 2024 - FFmpeg-trac - lists.ffmpeg.org

#11083(swresample:new): Converting multichannel audio in FLTP sample format to stereo in S16 attenuates volume unexpectedly
by FFmpeg 04 Jul '24

04 Jul '24

#11083: Converting multichannel audio in FLTP sample format to stereo in S16 attenuates volume unexpectedly -------------------------------------+------------------------------------- Reporter: Jiamin.X | Type: defect Status: new | Priority: important Component: swresample | Version: | unspecified Keywords: resampling | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- When converting multichannel audio in FLTP sample format to stereo in S16 sample format, volume is decreased unexpectedly. The original 6-channel audio input file in FLTP sample format: {{{ % ffprobe multich-audio.mp4 ffprobe version 6.0 Copyright (c) 2007-2023 the FFmpeg developers built with Apple clang version 15.0.0 (clang-1500.0.40.1) configuration: --prefix=/usr/local/Cellar/ffmpeg/6.0_2 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host- ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable- libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable- libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable- libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox libavutil 58. 2.100 / 58. 2.100 libavcodec 60. 3.100 / 60. 3.100 libavformat 60. 3.100 / 60. 3.100 libavdevice 60. 1.100 / 60. 1.100 libavfilter 9. 3.100 / 9. 3.100 libswscale 7. 1.100 / 7. 1.100 libswresample 4. 10.100 / 4. 10.100 libpostproc 57. 1.100 / 57. 1.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomdby1iso2mp41 encoder : www.aliyun.com - Media Transcoding Duration: 00:01:06.50, start: 0.000000, bitrate: 256 kb/s Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] Side data: audio service type: main }}} Converted from 6-channel in **fltp** to 2-chhanel in **flt**, output: **stereo-flt.mkv** {{{ % ffmpeg -i multich-audio.mp4 -ac 2 -c:a pcm_f32le stereo-flt.mkv Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Duration: 00:01:06.50, start: 0.000000, bitrate: 256 kb/s Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_f32le (native)) Output #0, matroska, to 'stereo-flt.mkv': Stream #0:0(und): Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, stereo, flt, 3072 kb/s (default) }}} Converted from 6-channel in **fltp** to 2-chhanel in **s16**, output: **stereo-s16.mkv** {{{ % ffmpeg -i multich-audio.mp4 -ac 2 -c:a pcm_s16le stereo-s16.mkv Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_s16le (native)) Output #0, matroska, to 'stereo-s16.mkv': Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s (default) }}} Converted from 6-channel in **fltp** to 2-chhanel in **s16**, with **-rematrix_maxval 1000**, output: **stereo-s16-rematrix_maxval-1000.mkv** {{{ % ffmpeg -i multich-audio.mp4 -rematrix_maxval 1000 -ac 2 -c:a pcm_s16le stereo-s16-rematrix_maxval-1000.mkv Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_s16le (native)) Output #0, matroska, to 'stereo-s16-rematrix_maxval-1000.mkv': Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s (default) }}} Using **volumedetect** to check the max and mean volumes of the original file and the 3 generated files above: 1. Volume statistics of **multich-audio.mp4** (the original file): {{{ % ffmpeg -i multich-audio.mp4 -af "volumedetect" -vn -sn -f null /dev/null Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0(und): Audio: pcm_s16le, 48000 Hz, 5.1(side), s16, 4608 kb/s (default) [Parsed_volumedetect_0 @ 0x7fd9a47052c0] n_samples: 19150848 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] mean_volume: -25.4 dB [Parsed_volumedetect_0 @ 0x7fd9a47052c0] max_volume: -1.9 dB [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_1db: 68 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_2db: 2022 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_3db: 3665 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_4db: 6371 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_5db: 10144 }}} 2. Volume statistics of **stereo-flt.mkv**: {{{ % ffmpeg -i stereo-flt.mkv -af "volumedetect" -vn -sn -f null /dev/null Input #0, matroska,webm, from 'stereo-flt.mkv': Stream #0:0: Audio: pcm_f32le, 48000 Hz, 2 channels, flt, 3072 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (pcm_f32le (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s (default) [Parsed_volumedetect_0 @ 0x7fce1e404200] n_samples: 6383616 [Parsed_volumedetect_0 @ 0x7fce1e404200] mean_volume: -21.6 dB [Parsed_volumedetect_0 @ 0x7fce1e404200] max_volume: 0.0 dB [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_0db: 1466 [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_1db: 1310 [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_2db: 3452 [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_3db: 4591 }}} 3. Volume statistics of **stereo-rematrix_maxval-1000.mkv**: {{{ % ffmpeg -i stereo-s16-rematrix_maxval-1000.mkv -af "volumedetect" -vn -sn -f null /dev/null Input #0, matroska,webm, from 'stereo-s16-rematrix_maxval-1000.mkv': Stream #0:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (pcm_s16le (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s (default) [Parsed_volumedetect_0 @ 0x7febc6a06180] n_samples: 6383616 [Parsed_volumedetect_0 @ 0x7febc6a06180] mean_volume: -21.6 dB [Parsed_volumedetect_0 @ 0x7febc6a06180] max_volume: 0.0 dB [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_0db: 1466 [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_1db: 1310 [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_2db: 3452 [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_3db: 4591 }}} 4. Volume statistics of **stereo-s16.mkv**: {{{ % ffmpeg -i stereo-s16.mkv -af "volumedetect" -vn -sn -f null /dev/null Input #0, matroska,webm, from 'stereo-s16.mkv': Stream #0:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (pcm_s16le (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s (default) [Parsed_volumedetect_0 @ 0x7fbf80a080c0] n_samples: 6383616 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] mean_volume: -29.3 dB [Parsed_volumedetect_0 @ 0x7fbf80a080c0] max_volume: -5.9 dB [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_5db: 21 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_6db: 294 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_7db: 598 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_8db: 831 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_9db: 1816 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_10db: 4223 }}} From the above, we can see that converting to **fltp** or **s16 with -rematrix_maxval 100** have the exact same max and mean volume, while converting to **s16** directly without setting **-rematrix_maxval** results in lot weaker volume. If I convert from 2-channel **flt** to 2-channel **s16**, the volume will not be affected at all. I checked the code related to the **rematrix_maxval** setting in **libswresample/rematrix.c** (shown below), we can see if **rematrix_maxval** is not manually set, it will be treated differently based on the input and output sample formats, basically for **s16** as either the input or output sample format, **rematrix_maxval** will be set to 1, which will affect the matrix params used in the later rematrix process, essentially attenuates the coefficients used for downmixing and cause volume attenuation as a result. My confusion is that why we have to check the output sample format and adjust this **rematrix_maxval** accordingly before dowxmixing, it looks to me that only the input sample format will affect the rematrix/downmix process, because rematrix/downmix will operate on the input data the same way regardless of the output sample format. If I am right, we may need to remove the **av_get_packed_sample_fmt(s->out_sample_fmt) < AV_SAMPLE_FMT_FLT** check in the fowlloing code (If this is correct, I may send a pull request later): {{{ av_cold static int auto_matrix(SwrContext *s) { double maxval; int ret; if (s->rematrix_maxval > 0) { maxval = s->rematrix_maxval; } else if ( av_get_packed_sample_fmt(s->out_sample_fmt) < AV_SAMPLE_FMT_FLT || av_get_packed_sample_fmt(s->int_sample_fmt) < AV_SAMPLE_FMT_FLT) { maxval = 1.0; } else maxval = INT_MAX; ... } av_cold int swr_build_matrix(uint64_t in_ch_layout_param, uint64_t out_ch_layout_param, double center_mix_level, double surround_mix_level, double lfe_mix_level, double maxval, double rematrix_volume, double *matrix_param, int stride, enum AVMatrixEncoding matrix_encoding, void *log_context) { ... if(maxcoef > maxval || rematrix_volume < 0){ maxcoef /= maxval; for(i=0; i<SWR_CH_MAX; i++) for(j=0; j<SWR_CH_MAX; j++){ matrix_param[stride*i + j] /= maxcoef; } } .... } }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/11083> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 2

#11082(swresample:new): Converting multichannel audio in FLTP sample format to stereo in S16 attenuates volume unexpectedly
by FFmpeg 04 Jul '24

04 Jul '24

#11082: Converting multichannel audio in FLTP sample format to stereo in S16 attenuates volume unexpectedly -------------------------------------+------------------------------------- Reporter: Jiamin.X | Type: defect Status: new | Priority: important Component: swresample | Version: | unspecified Keywords: resampling | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- When converting multichannel audio in FLTP sample format to stereo in S16 sample format, volume is decreased unexpectedly. The original 6-channel audio input file in FLTP sample format: {{{ % ffprobe multich-audio.mp4 ffprobe version 6.0 Copyright (c) 2007-2023 the FFmpeg developers built with Apple clang version 15.0.0 (clang-1500.0.40.1) configuration: --prefix=/usr/local/Cellar/ffmpeg/6.0_2 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host- ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable- libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable- libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable- libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable- libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox libavutil 58. 2.100 / 58. 2.100 libavcodec 60. 3.100 / 60. 3.100 libavformat 60. 3.100 / 60. 3.100 libavdevice 60. 1.100 / 60. 1.100 libavfilter 9. 3.100 / 9. 3.100 libswscale 7. 1.100 / 7. 1.100 libswresample 4. 10.100 / 4. 10.100 libpostproc 57. 1.100 / 57. 1.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomdby1iso2mp41 encoder : www.aliyun.com - Media Transcoding Duration: 00:01:06.50, start: 0.000000, bitrate: 256 kb/s Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] Side data: audio service type: main }}} Converted from 6-channel in **fltp** to 2-chhanel in **flt**, output: **stereo-flt.mkv** {{{ % ffmpeg -i multich-audio.mp4 -ac 2 -c:a pcm_f32le stereo-flt.mkv Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Duration: 00:01:06.50, start: 0.000000, bitrate: 256 kb/s Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_f32le (native)) Output #0, matroska, to 'stereo-flt.mkv': Stream #0:0(und): Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, stereo, flt, 3072 kb/s (default) }}} Converted from 6-channel in **fltp** to 2-chhanel in **s16**, output: **stereo-s16.mkv** {{{ % ffmpeg -i multich-audio.mp4 -ac 2 -c:a pcm_s16le stereo-s16.mkv Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_s16le (native)) Output #0, matroska, to 'stereo-s16.mkv': Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s (default) }}} Converted from 6-channel in **fltp** to 2-chhanel in **s16**, with **-rematrix_maxval 1000**, output: **stereo-s16-rematrix_maxval-1000.mkv** {{{ % ffmpeg -i multich-audio.mp4 -rematrix_maxval 1000 -ac 2 -c:a pcm_s16le stereo-s16-rematrix_maxval-1000.mkv Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_s16le (native)) Output #0, matroska, to 'stereo-s16-rematrix_maxval-1000.mkv': Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s (default) }}} Using **volumedetect** to check the max and mean volumes of the original file and the 3 generated files above: 1. Volume statistics of **multich-audio.mp4** (the original file): {{{ % ffmpeg -i multich-audio.mp4 -af "volumedetect" -vn -sn -f null /dev/null Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'multich-audio.mp4': Stream #0:0[0x1](und): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 256 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (eac3 (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0(und): Audio: pcm_s16le, 48000 Hz, 5.1(side), s16, 4608 kb/s (default) [Parsed_volumedetect_0 @ 0x7fd9a47052c0] n_samples: 19150848 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] mean_volume: -25.4 dB [Parsed_volumedetect_0 @ 0x7fd9a47052c0] max_volume: -1.9 dB [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_1db: 68 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_2db: 2022 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_3db: 3665 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_4db: 6371 [Parsed_volumedetect_0 @ 0x7fd9a47052c0] histogram_5db: 10144 }}} 2. Volume statistics of **stereo-flt.mkv**: {{{ % ffmpeg -i stereo-flt.mkv -af "volumedetect" -vn -sn -f null /dev/null Input #0, matroska,webm, from 'stereo-flt.mkv': Stream #0:0: Audio: pcm_f32le, 48000 Hz, 2 channels, flt, 3072 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (pcm_f32le (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s (default) [Parsed_volumedetect_0 @ 0x7fce1e404200] n_samples: 6383616 [Parsed_volumedetect_0 @ 0x7fce1e404200] mean_volume: -21.6 dB [Parsed_volumedetect_0 @ 0x7fce1e404200] max_volume: 0.0 dB [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_0db: 1466 [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_1db: 1310 [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_2db: 3452 [Parsed_volumedetect_0 @ 0x7fce1e404200] histogram_3db: 4591 }}} 3. Volume statistics of **stereo-rematrix_maxval-1000.mkv**: {{{ % ffmpeg -i stereo-s16-rematrix_maxval-1000.mkv -af "volumedetect" -vn -sn -f null /dev/null Input #0, matroska,webm, from 'stereo-s16-rematrix_maxval-1000.mkv': Stream #0:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (pcm_s16le (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s (default) [Parsed_volumedetect_0 @ 0x7febc6a06180] n_samples: 6383616 [Parsed_volumedetect_0 @ 0x7febc6a06180] mean_volume: -21.6 dB [Parsed_volumedetect_0 @ 0x7febc6a06180] max_volume: 0.0 dB [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_0db: 1466 [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_1db: 1310 [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_2db: 3452 [Parsed_volumedetect_0 @ 0x7febc6a06180] histogram_3db: 4591 }}} 4. Volume statistics of **stereo-s16.mkv**: {{{ % ffmpeg -i stereo-s16.mkv -af "volumedetect" -vn -sn -f null /dev/null Input #0, matroska,webm, from 'stereo-s16.mkv': Stream #0:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s (default) Stream mapping: Stream #0:0 -> #0:0 (pcm_s16le (native) -> pcm_s16le (native)) Output #0, null, to '/dev/null': Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s (default) [Parsed_volumedetect_0 @ 0x7fbf80a080c0] n_samples: 6383616 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] mean_volume: -29.3 dB [Parsed_volumedetect_0 @ 0x7fbf80a080c0] max_volume: -5.9 dB [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_5db: 21 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_6db: 294 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_7db: 598 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_8db: 831 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_9db: 1816 [Parsed_volumedetect_0 @ 0x7fbf80a080c0] histogram_10db: 4223 }}} From the above, we can see that converting to **fltp** or **s16 with -rematrix_maxval 100** have the exact same max and mean volume, while converting to **s16** directly without setting **-rematrix_maxval** results in lot weaker volume. If I convert from 2-channel **flt** to 2-channel **s16**, the volume will not be affected at all. I checked the code related to the **rematrix_maxval** setting in **libswresample/rematrix.c** (shown below), we can see if **rematrix_maxval** is not manually set, it will be treated differently based on the input and output sample formats, basically for **s16** as either the input or output sample format, **rematrix_maxval** will be set to 1, which will affect the matrix params used in the later rematrix process, essentially attenuates the coefficients used for downmixing and cause volume attenuation as a result. My confusion is that why we have to check the output sample format and adjust this **rematrix_maxval** accordingly before dowxmixing, it looks to me that only the input sample format will affect the rematrix/downmix process, because rematrix/downmix will operate on the input data the same way regardless of the output sample format. If I am right, we may need to remove the **av_get_packed_sample_fmt(s->out_sample_fmt) < AV_SAMPLE_FMT_FLT** check in the fowlloing code (If this is correct, I may send a pull request later): {{{ av_cold static int auto_matrix(SwrContext *s) { double maxval; int ret; if (s->rematrix_maxval > 0) { maxval = s->rematrix_maxval; } else if ( av_get_packed_sample_fmt(s->out_sample_fmt) < AV_SAMPLE_FMT_FLT || av_get_packed_sample_fmt(s->int_sample_fmt) < AV_SAMPLE_FMT_FLT) { maxval = 1.0; } else maxval = INT_MAX; ... } av_cold int swr_build_matrix(uint64_t in_ch_layout_param, uint64_t out_ch_layout_param, double center_mix_level, double surround_mix_level, double lfe_mix_level, double maxval, double rematrix_volume, double *matrix_param, int stride, enum AVMatrixEncoding matrix_encoding, void *log_context) { ... if(maxcoef > maxval || rematrix_volume < 0){ maxcoef /= maxval; for(i=0; i<SWR_CH_MAX; i++) for(j=0; j<SWR_CH_MAX; j++){ matrix_param[stride*i + j] /= maxcoef; } } .... } }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/11082> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 1

#11081(undetermined:new): Mov to MP4 produces weird artifacts if mov has an alpha channel (transparency)
by FFmpeg 03 Jul '24

03 Jul '24

#11081: Mov to MP4 produces weird artifacts if mov has an alpha channel (transparency) -------------------------------------+------------------------------------- Reporter: Nommalorel | Type: art Status: new | Priority: normal Component: | Version: 7.0 undetermined | Keywords: art , | Blocked By: animation , render , conversion , | movtomp4 , transparent , artifact | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Summary of the bug: When i try to convert my animation .mov file that i rendered from spine2d that has a transparent background to an .mp4 file ffmpeg creates weird artifacts on and around the character. First frame that is converted by adobe media encoder: https://i.ibb.co/mTyZ3W5/adb-md-enc-f1.png First frame that is converted by ffmpeg (or any FOSS converter that uses ffmpeg as a base) : https://i.ibb.co/hR4PmVz/ffmpeg-f1.png How to reproduce: {{{ Render a mov file from any animation or video editing program with a video that has an object or a character at the middle but has a transparent background. Then convert with the code stated below. % ffmpeg -i {inname}.mov {outname}.mp4 ffmpeg version ffmpeg version 7.0.1-full_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers built with gcc 13.2.0 (Rev5, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --disable- w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libxevd --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxeve --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable- liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable- amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable- libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable- libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable- libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint libavutil 59. 8.100 / 59. 8.100 libavcodec 61. 3.100 / 61. 3.100 libavformat 61. 1.100 / 61. 1.100 libavdevice 61. 1.100 / 61. 1.100 libavfilter 10. 1.100 / 10. 1.100 libswscale 8. 1.100 / 8. 1.100 libswresample 5. 1.100 / 5. 1.100 libpostproc 58. 1.100 / 58. 1.100 built on ... Windows 10 }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/11081> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 0

#10376(undetermined:new): Consider replacing long / unsigned long data types?
by FFmpeg 02 Jul '24

02 Jul '24

#10376: Consider replacing long / unsigned long data types? -------------------------------------+------------------------------------- Reporter: SuRGeoNix | Type: | enhancement Status: new | Priority: wish Component: | Version: undetermined | unspecified Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Those data types are platform specific (eg 8-byte in Mac, 4-byte in Win and 4-byte in x86 Linux but 8-byte in x64 Linux) and can cause issues when used from higher libraries (eg. see https://github.com/Ruslan-B/FFmpeg.AutoGen/issues/217) with struct size/alignments. They shouldn't be many (at least for public API) but the most important one, is the 'unsigned long checksum' in AVIOContext. (Just wondering, any actual reason of their existence?) -- Ticket URL: <https://trac.ffmpeg.org/ticket/10376> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 2

#11079(undetermined:new): Incorrect handling of mbedtls_x509_crt_parse_file return code
by FFmpeg 01 Jul '24

01 Jul '24

#11079: Incorrect handling of mbedtls_x509_crt_parse_file return code -------------------------------------+------------------------------------- Reporter: Skybound | Type: defect Status: new | Priority: normal Component: | Version: undetermined | unspecified Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Summary of the bug: When using mbed TLS and loading a CA file, loading will fail if any of the certificates in the file cannot be parsed by mbed. For example, when having a name constraint as a critical extension which is not currently supported by mbed (https://github.com/Mbed-TLS/mbedtls/issues/8759) This is due to the usage of `mbedtls_x509_crt_parse_file`, and the check to make sure this is non-zero within `libavformat/tls_mbedtls.c`. This should check if the return code is < 0, with values greater than 0 deemed acceptable. A value greater than 0 overall represents a success, just that certain certificates from the file were skipped. How to reproduce: Create a ca bundle with various system trust certificates + a certificate with a name constraint. I have attached an example. {{{ % ffplay -i https://google.com -tls_verify 1 -cafile /tmp/ca-bundle.crt ... mbedtls_x509_crt_parse_file for CA cert returned 1 ... }}} Duplicating the name constrained certificate in the CA file changes and re-running the command changes the return code to 2. Remove the name constrained certificate from the CA file (its the first one marked as such) and re-run the command. This time it will give an invalid data error and no longer an SSL error. (this is expected as we passed in google.com not a playable URL, and is not an issue being raised) {{{ $ ffplay -version ffplay version n7.0.1 Copyright (c) 2003-2024 the FFmpeg developers built with gcc 14.1.1 (GCC) 20240522 configuration: --prefix=/usr --disable-debug --disable-static --disable- stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gpl --enable- ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libdvdnav --enable-libdvdread --enable-libfreetype --enable-libfribidi --enable-libgsm --enable- libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable- libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable- libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-mbedtls --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable- shared --enable-vapoursynth --enable-version3 --enable-vulkan libavutil 59. 8.100 / 59. 8.100 libavcodec 61. 3.100 / 61. 3.100 libavformat 61. 1.100 / 61. 1.100 libavdevice 61. 1.100 / 61. 1.100 libavfilter 10. 1.100 / 10. 1.100 libswscale 8. 1.100 / 8. 1.100 libswresample 5. 1.100 / 5. 1.100 libpostproc 58. 1.100 / 58. 1.100 }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/11079> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 3

#10829(undetermined:new): -skip_frame nokey is ignored for vp9/webm
by FFmpeg 01 Jul '24

01 Jul '24

#10829: -skip_frame nokey is ignored for vp9/webm -------------------------------------+------------------------------------- Reporter: Konstantin | Type: defect Vlasov | Status: new | Priority: normal Component: | Version: 6.1 undetermined | Keywords: nokey | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- How to reproduce: {{{ > ffprobe.exe -hide_banner -show_frames -skip_frame nokey -select_streams v test.webm >test.txt Input #0, matroska,webm, from 'test.webm': Metadata: ENCODER : Lavf60.3.100 Duration: 00:00:33.25, start: 0.000000, bitrate: 12521 kb/s Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709), 3840x2160, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 1k tbn (default) Metadata: DURATION : 00:00:33.233000000 Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default) Metadata: DURATION : 00:00:33.248000000 }}} The resulting file `test.txt` contains information about all the frames, not just the key frames. For the test I took some video downloaded from YouTube (specifically, this one: https://www.youtube.com/watch?v=nUiHDtRZf44 , using the format 315+251). Reproduced in Windows builds from gyan.dev, versions 6.1.1 and master build 2024-01-24-git-00b288da73: {{{ >C:\Programs\ffmpeg-nightly\bin\ffprobe.exe -version ffprobe version 2024-01-24-git-00b288da73-full_build-www.gyan.dev Copyright (c) 2007-2024 the FFmpeg developers built with gcc 12.2.0 (Rev10, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --pkg- config=pkgconf --disable-w32threads --disable-autodetect --enable- fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable- librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable- libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable- libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable- libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable- libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable- libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable- libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable- libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable- libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable- libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable- libsoxr --enable-chromaprint libavutil 58. 36.101 / 58. 36.101 libavcodec 60. 38.100 / 60. 38.100 libavformat 60. 20.100 / 60. 20.100 libavdevice 60. 4.100 / 60. 4.100 libavfilter 9. 17.100 / 9. 17.100 libswscale 7. 6.100 / 7. 6.100 libswresample 4. 13.100 / 4. 13.100 libpostproc 57. 4.100 / 57. 4.100 >C:\Programs\ffmpeg-6.1.1\bin\ffprobe.exe -version ffprobe version 6.1.1-full_build-www.gyan.dev Copyright (c) 2007-2023 the FFmpeg developers built with gcc 12.2.0 (Rev10, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --pkg- config=pkgconf --disable-w32threads --disable-autodetect --enable- fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable- librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable- libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable- libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable- libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable- libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable- libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable- libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable- libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable- libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable- libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable- libsoxr --enable-chromaprint libavutil 58. 29.100 / 58. 29.100 libavcodec 60. 31.102 / 60. 31.102 libavformat 60. 16.100 / 60. 16.100 libavdevice 60. 3.100 / 60. 3.100 libavfilter 9. 12.100 / 9. 12.100 libswscale 7. 5.100 / 7. 5.100 libswresample 4. 12.100 / 4. 12.100 libpostproc 57. 3.100 / 57. 3.100 }}} -- Ticket URL: <https://trac.ffmpeg.org/ticket/10829> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker

1 7