[FFmpeg-trac] #6006(avfilter:reopened): FPS filter stutters when converting framerate (was: FPS filter not selecting right frames from input)
FFmpeg
trac at avcodec.org
Wed Apr 25 22:09:25 EEST 2018
#6006: FPS filter stutters when converting framerate
------------------------------------+------------------------------------
Reporter: Misaki | Owner:
Type: defect | Status: reopened
Priority: normal | Component: avfilter
Version: git-master | Resolution:
Keywords: fps | Blocked By:
Blocking: | Reproduced by developer: 1
Analyzed by developer: 0 |
------------------------------------+------------------------------------
Changes (by Misaki):
* status: closed => reopened
* resolution: fixed =>
Comment:
Reproduced on latest git build from https://johnvansickle.com/ffmpeg/.
{{{
$ ./ffmpeg -filter_complex
color=white:r=30,format=gray,fade=in:0:235,settb=1/1000,fps=15:start_time=0:round=near,showinfo
-t 0.2 -f null -
ffmpeg version N-45834-ga12899ad9-static https://johnvansickle.com/ffmpeg/
Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
configuration: --enable-gpl --enable-version3 --enable-static --disable-
debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio
--cc=gcc-6 --enable-libxml2 --enable-fontconfig --enable-frei0r --enable-
gnutls --enable-gray --enable-libaom --enable-libfribidi --enable-libass
--enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
--enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
--enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus
--enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-
libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2
--enable-libxvid --enable-libzimg
libavutil 56. 15.100 / 56. 15.100
libavcodec 58. 19.100 / 58. 19.100
libavformat 58. 13.100 / 58. 13.100
libavdevice 58. 4.100 / 58. 4.100
libavfilter 7. 19.100 / 7. 19.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100
Stream mapping:
showinfo -> Stream #0:0 (wrapped_avframe)
Press [q] to stop, [?] for help
[Parsed_showinfo_5 @ 0x4f46940] config in time_base: 1/15, frame_rate:
15/1
[Parsed_showinfo_5 @ 0x4f46940] config out time_base: 0/0, frame_rate: 0/0
Output #0, null, to 'pipe:':
Metadata:
encoder : Lavf58.13.100
Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3],
q=2-31, 200 kb/s, 15 fps, 15 tbn, 15 tbc (default)
Metadata:
encoder : Lavc58.19.100 wrapped_avframe
[Parsed_showinfo_5 @ 0x4f46940] n: 0 pts: 0 pts_time:0 pos:
-1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:79FA842D
plane_checksum:[79FA842D] mean:[1] stdev:[0.0]
[Parsed_showinfo_5 @ 0x4f46940] n: 1 pts: 1 pts_time:0.0666667 pos:
-1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:F3F40869
plane_checksum:[F3F40869] mean:[2] stdev:[0.0]
[Parsed_showinfo_5 @ 0x4f46940] n: 2 pts: 2 pts_time:0.133333 pos:
-1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:E7F710D2
plane_checksum:[E7F710D2] mean:[4] stdev:[0.0]
[Parsed_showinfo_5 @ 0x4f46940] n: 3 pts: 3 pts_time:0.2 pos:
-1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:56039D68
plane_checksum:[56039D68] mean:[7] stdev:[0.0]
frame= 3 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.20 bitrate=N/A
speed=8.27x
video:2kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: unknown
}}}
Frame number (from 0) is the mean:[x] value. With round=near, it's
[1,2,4,7]. With round=up and round=inf, it's [0,1,4,6]. With round=zero
and round=down, it's [1,4,5,7]. With start_time=1 and '-t 1.2', it's
[31,34,35,37]; however, with 'start_time=0.11:round=near' and '-t 0.31',
it's [4,6,8,10].
This report would be much better if I offered a patch, but I don't have
the expertise.
When framerate is halved, the fps filter uses an interval or point that's
midway between the start point and the next input frame. I don't know how
it's done mathematically or in the code, but this is how it works. Due to
timebase rounding, sometimes the next frame is closer, and sometimes the
previous one is.
With input framerate = output framerate, the fps filter will still
duplicate and drop frames with some rounding methods; in this case,
'fps=30:start_time=0:round=down' leads to [1,1,2,4,4,5,7].
The frame duplication in the first case can be avoided by using a
start_time offset that exceeds the variation in frames. But if YouTube can
get this wrong, normal users can as well. At least one of the rounding
methods should work without adjusting the start time, and it might as well
be 'near'.
Conceptually, the interval should be centered on the first frame, not on
the average between first and last frames of the input interval. So for
input=30fps and output=30fps, the interval is [-1/60,1/60]. Frame at 0 is
closest. For output=15fps, interval is [-1/30,1/30]. Frame at 0 is still
closest. Second interval for 15 fps is [1/30,3/30], and the frame at
pts=67:pts_time=0.067 is closest. There's one very obvious candidate
instead of two very similar candidate frames.
This can still lead to problems if the input timebase isn't divisible by
framerate, and the offset gradually or suddenly jumps so that the decision
point is equidistant from two frames. Could happen with variable framerate
video that has been edited; when concatenating two videos with ffmpeg,
which leads to odd offsets due to audio having a different length than
video, like the second clip being 0.01 sec early or late; or when
converting ~59.97 fps video to 30 fps.
The muxers in ffmpeg will add or drop a frame based on an acceptable
offset, like 0.5 or 1.5 times the distance between frames based on
framerate from -r [rate]. The fps filter could try to select the next
input frame to use for output based on the previous one used, by
attempting to moderate the high-frequency variations introduced by
timebase rounding, but this would be a more extensive patch and just
making round=near work for halving framerate would a good fix by itself.
--
Ticket URL: <https://trac.ffmpeg.org/ticket/6006#comment:4>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list