[FFmpeg-trac] #10018(undetermined:new): Duration estimation regression for h264 in mov
FFmpeg
trac at avcodec.org
Thu Nov 10 12:13:13 EET 2022
#10018: Duration estimation regression for h264 in mov
-------------------------------------+-------------------------------------
Reporter: Peter Z | Owner: (none)
Type: defect | Status: new
Priority: normal | Component:
| undetermined
Version: unspecified | Resolution:
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Description changed by Peter Z:
Old description:
> Summary of the bug:
> There appears to be a regression with guesstimation of packet durations
> in ffmpeg v5 compared to v4.4.1. Tested on latest master. The problem can
> be seen both with ffprobe and by doing a remux of the content, with the
> remux the total file duration will be affected.
>
> How to reproduce:
> Create a test file with a frame rate of 30fps with a timescale double
> that.
> {{{
> % ./ffmpeg -f lavfi -i testsrc=duration=10:size=1728x720:rate=30 -c:v
> libx264 -b:v 1000000 -video_track_timescale 60 -movie_timescale 60
> repro.mp4
> ffmpeg version N-109048-g65f3bc9e7e Copyright (c) 2000-2022 the FFmpeg
> developers
> built with Apple clang version 14.0.0 (clang-1400.0.29.202)
> configuration: --enable-static --enable-ffmpeg --enable-ffprobe
> --disable-shared --enable-gpl --enable-gray --enable-nonfree --enable-
> iconv --enable-libxml2 --enable-libmp3lame --enable-libfdk-aac --enable-
> libvorbis --enable-libopus --enable-libtheora --enable-libvpx --enable-
> libx264 --enable-libx265 --enable-libxml2 --disable-stripping --enable-
> libfreetype --enable-libsoxr --extra-cflags=-I/opt/homebrew/include
> --extra-ldflags=-L/opt/homebrew/lib
> libavutil 57. 42.100 / 57. 42.100
> libavcodec 59. 52.100 / 59. 52.100
> libavformat 59. 34.101 / 59. 34.101
> libavdevice 59. 8.101 / 59. 8.101
> libavfilter 8. 50.100 / 8. 50.100
> libswscale 6. 8.112 / 6. 8.112
> libswresample 4. 9.100 / 4. 9.100
> libpostproc 56. 7.100 / 56. 7.100
> Input #0, lavfi, from 'testsrc=duration=10:size=1728x720:rate=30':
> Duration: N/A, start: 0.000000, bitrate: N/A
> Stream #0:0: Video: wrapped_avframe, rgb24, 1728x720 [SAR 1:1 DAR
> 12:5], 30 fps, 30 tbr, 30 tbn
> Stream mapping:
> Stream #0:0 -> #0:0 (wrapped_avframe (native) -> h264 (libx264))
> Press [q] to stop, [?] for help
> [libx264 @ 0x1452061e0] using SAR=1/1
> [libx264 @ 0x1452061e0] using cpu capabilities: ARMv8 NEON
> [libx264 @ 0x1452061e0] profile High 4:4:4 Predictive, level 3.2, 4:4:4,
> 8-bit
> [libx264 @ 0x1452061e0] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC
> codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options:
> cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1
> psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1
> cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=15
> lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0
> bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
> b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
> scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=1000
> ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
> Output #0, mp4, to 'repro.mp4':
> Metadata:
> encoder : Lavf59.34.101
> Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv444p(tv, progressive),
> 1728x720 [SAR 1:1 DAR 12:5], q=2-31, 1000 kb/s, 30 fps, 60 tbn
> Metadata:
> encoder : Lavc59.52.100 libx264
> Side data:
> cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: N/A
> frame= 300 fps=0.0 q=-1.0 Lsize= 547kB time=00:00:09.90 bitrate=
> 452.9kbits/s speed=11.6x
> video:543kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: 0.804443%
> [libx264 @ 0x1452061e0] frame I:2 Avg QP:11.65 size: 12071
> [libx264 @ 0x1452061e0] frame P:76 Avg QP: 4.77 size: 2199
> [libx264 @ 0x1452061e0] frame B:222 Avg QP: 4.38 size: 1640
> [libx264 @ 0x1452061e0] consecutive B-frames: 1.3% 0.0% 0.0% 98.7%
> [libx264 @ 0x1452061e0] mb I I16..4: 72.3% 23.5% 4.2%
> [libx264 @ 0x1452061e0] mb P I16..4: 2.1% 0.5% 0.6% P16..4: 3.1%
> 0.5% 0.1% 0.0% 0.0% skip:93.0%
> [libx264 @ 0x1452061e0] mb B I16..4: 0.0% 0.1% 0.0% B16..8: 3.6%
> 0.2% 0.0% direct: 1.6% skip:94.4% L0:52.6% L1:46.7% BI: 0.7%
> [libx264 @ 0x1452061e0] final ratefactor: 0.18
> [libx264 @ 0x1452061e0] 8x8 transform intra:21.2% inter:84.3%
> [libx264 @ 0x1452061e0] coded y,u,v intra: 6.2% 7.1% 6.3% inter: 0.1%
> 1.3% 1.2%
> [libx264 @ 0x1452061e0] i16 v,h,dc,p: 89% 9% 1% 2%
> [libx264 @ 0x1452061e0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 62% 4% 33% 0%
> 0% 0% 0% 0% 0%
> [libx264 @ 0x1452061e0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 47% 34% 0%
> 0% 0% 0% 0% 0%
> [libx264 @ 0x1452061e0] Weighted P-Frames: Y:0.0% UV:0.0%
> [libx264 @ 0x1452061e0] ref P L0: 84.6% 0.8% 10.9% 3.6%
> [libx264 @ 0x1452061e0] ref B L0: 93.8% 6.2% 0.1%
> [libx264 @ 0x1452061e0] ref B L1: 98.2% 1.8%
> [libx264 @ 0x1452061e0] kb/s:444.27
>
> }}}
> Check the duration of the packets with ffprobe:
> {{{
> % ffprobe -show_packets repro.mp4
> }}}
> With ffmpeg 4.4.1 this would show packet duration as 2 (2/60) but with
> ffmpeg 5.x all the packets get duration 1 (1/60).
>
> The consequences of this can also be observed by remuxing the file:
> {{{
> % ffmpeg -i repro.mp4 -c copy remux.mp4
> --- full ffprobe output in attached file ---
> }}}
>
> Now the probed duration is no longer 10s but 1/60 less 9.983333.
>
> I think this problem might have appeared after H264 parsing was
> introduced with this commit:
> 8a3f8afa4e46011e9c5849f8e0d57ec9b53deef7
>
> At least I have observed that during the execution of the
> compute_frame_duration function there is in ffmpeg5 a valid pointer to a
> AVCodecParserContext set but not in version 4.4.1 of ffmpeg.
> https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/demux.c#L668
> Thus the heuristic for guessing the frame duration is now taking the
> second branch and going purely on timebase.
>
> Patches should be submitted to the ffmpeg-devel mailing list and not this
> bug tracker.
New description:
Summary of the bug:
There appears to be a regression with guesstimation of packet durations in
ffmpeg v5 compared to v4.4.1. Tested on latest master. The problem can be
seen both with ffprobe and by doing a remux of the content, with the remux
the total file duration will be affected.
How to reproduce:
Create a test file with a frame rate of 30fps with a timescale double
that.
{{{
% ./ffmpeg -f lavfi -i testsrc=duration=10:size=1728x720:rate=30 -c:v
libx264 -b:v 1000000 -video_track_timescale 60 -movie_timescale 60
repro.mp4
ffmpeg version N-109048-g65f3bc9e7e Copyright (c) 2000-2022 the FFmpeg
developers
built with Apple clang version 14.0.0 (clang-1400.0.29.202)
configuration: --enable-static --enable-ffmpeg --enable-ffprobe
--disable-shared --enable-gpl --enable-gray --enable-nonfree --enable-
iconv --enable-libxml2 --enable-libmp3lame --enable-libfdk-aac --enable-
libvorbis --enable-libopus --enable-libtheora --enable-libvpx --enable-
libx264 --enable-libx265 --enable-libxml2 --disable-stripping --enable-
libfreetype --enable-libsoxr --extra-cflags=-I/opt/homebrew/include
--extra-ldflags=-L/opt/homebrew/lib
libavutil 57. 42.100 / 57. 42.100
libavcodec 59. 52.100 / 59. 52.100
libavformat 59. 34.101 / 59. 34.101
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 50.100 / 8. 50.100
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100
Input #0, lavfi, from 'testsrc=duration=10:size=1728x720:rate=30':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: wrapped_avframe, rgb24, 1728x720 [SAR 1:1 DAR 12:5],
30 fps, 30 tbr, 30 tbn
Stream mapping:
Stream #0:0 -> #0:0 (wrapped_avframe (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x1452061e0] using SAR=1/1
[libx264 @ 0x1452061e0] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x1452061e0] profile High 4:4:4 Predictive, level 3.2, 4:4:4,
8-bit
[libx264 @ 0x1452061e0] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC
codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options:
cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1
cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=15
lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=1000
ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'repro.mp4':
Metadata:
encoder : Lavf59.34.101
Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv444p(tv, progressive),
1728x720 [SAR 1:1 DAR 12:5], q=2-31, 1000 kb/s, 30 fps, 60 tbn
Metadata:
encoder : Lavc59.52.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: N/A
frame= 300 fps=0.0 q=-1.0 Lsize= 547kB time=00:00:09.90 bitrate=
452.9kbits/s speed=11.6x
video:543kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 0.804443%
[libx264 @ 0x1452061e0] frame I:2 Avg QP:11.65 size: 12071
[libx264 @ 0x1452061e0] frame P:76 Avg QP: 4.77 size: 2199
[libx264 @ 0x1452061e0] frame B:222 Avg QP: 4.38 size: 1640
[libx264 @ 0x1452061e0] consecutive B-frames: 1.3% 0.0% 0.0% 98.7%
[libx264 @ 0x1452061e0] mb I I16..4: 72.3% 23.5% 4.2%
[libx264 @ 0x1452061e0] mb P I16..4: 2.1% 0.5% 0.6% P16..4: 3.1%
0.5% 0.1% 0.0% 0.0% skip:93.0%
[libx264 @ 0x1452061e0] mb B I16..4: 0.0% 0.1% 0.0% B16..8: 3.6%
0.2% 0.0% direct: 1.6% skip:94.4% L0:52.6% L1:46.7% BI: 0.7%
[libx264 @ 0x1452061e0] final ratefactor: 0.18
[libx264 @ 0x1452061e0] 8x8 transform intra:21.2% inter:84.3%
[libx264 @ 0x1452061e0] coded y,u,v intra: 6.2% 7.1% 6.3% inter: 0.1% 1.3%
1.2%
[libx264 @ 0x1452061e0] i16 v,h,dc,p: 89% 9% 1% 2%
[libx264 @ 0x1452061e0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 62% 4% 33% 0% 0%
0% 0% 0% 0%
[libx264 @ 0x1452061e0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 47% 34% 0% 0%
0% 0% 0% 0%
[libx264 @ 0x1452061e0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x1452061e0] ref P L0: 84.6% 0.8% 10.9% 3.6%
[libx264 @ 0x1452061e0] ref B L0: 93.8% 6.2% 0.1%
[libx264 @ 0x1452061e0] ref B L1: 98.2% 1.8%
[libx264 @ 0x1452061e0] kb/s:444.27
}}}
Check the duration of the packets with ffprobe:
{{{
% ffprobe -show_packets repro.mp4
--- full ffprobe output in attached file ---
}}}
With ffmpeg 4.4.1 this would show packet duration as 2 (2/60) but with
ffmpeg 5.x all the packets get duration 1 (1/60).
The consequences of this can also be observed by remuxing the file:
{{{
% ffmpeg -i repro.mp4 -c copy remux.mp4
}}}
Now the probed duration is no longer 10s but 1/60 less 9.983333.
I think this problem might have appeared after H264 parsing was introduced
with this commit:
8a3f8afa4e46011e9c5849f8e0d57ec9b53deef7
At least I have observed that during the execution of the
compute_frame_duration function there is in ffmpeg5 a valid pointer to a
AVCodecParserContext set but not in version 4.4.1 of ffmpeg.
https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/demux.c#L668
Thus the heuristic for guessing the frame duration is now taking the
second branch and going purely on timebase.
Patches should be submitted to the ffmpeg-devel mailing list and not this
bug tracker.
--
--
Ticket URL: <https://trac.ffmpeg.org/ticket/10018#comment:7>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list