[FFmpeg-trac] #6006(avfilter:reopened): FPS filter stutters when converting framerate (was: FPS filter not selecting right frames from input)

FFmpeg trac at avcodec.org
Wed Apr 25 22:09:25 EEST 2018


#6006: FPS filter stutters when converting framerate
------------------------------------+------------------------------------
             Reporter:  Misaki      |                    Owner:
                 Type:  defect      |                   Status:  reopened
             Priority:  normal      |                Component:  avfilter
              Version:  git-master  |               Resolution:
             Keywords:  fps         |               Blocked By:
             Blocking:              |  Reproduced by developer:  1
Analyzed by developer:  0           |
------------------------------------+------------------------------------
Changes (by Misaki):

 * status:  closed => reopened
 * resolution:  fixed =>


Comment:

 Reproduced on latest git build from https://johnvansickle.com/ffmpeg/.

 {{{
 $  ./ffmpeg -filter_complex
 color=white:r=30,format=gray,fade=in:0:235,settb=1/1000,fps=15:start_time=0:round=near,showinfo
 -t 0.2  -f null -

 ffmpeg version N-45834-ga12899ad9-static https://johnvansickle.com/ffmpeg/
 Copyright (c) 2000-2018 the FFmpeg developers
   built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
   configuration: --enable-gpl --enable-version3 --enable-static --disable-
 debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio
 --cc=gcc-6 --enable-libxml2 --enable-fontconfig --enable-frei0r --enable-
 gnutls --enable-gray --enable-libaom --enable-libfribidi --enable-libass
 --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb
 --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband
 --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus
 --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-
 libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2
 --enable-libxvid --enable-libzimg
   libavutil      56. 15.100 / 56. 15.100
   libavcodec     58. 19.100 / 58. 19.100
   libavformat    58. 13.100 / 58. 13.100
   libavdevice    58.  4.100 / 58.  4.100
   libavfilter     7. 19.100 /  7. 19.100
   libswscale      5.  2.100 /  5.  2.100
   libswresample   3.  2.100 /  3.  2.100
   libpostproc    55.  2.100 / 55.  2.100
 Stream mapping:
   showinfo -> Stream #0:0 (wrapped_avframe)
 Press [q] to stop, [?] for help
 [Parsed_showinfo_5 @ 0x4f46940] config in time_base: 1/15, frame_rate:
 15/1
 [Parsed_showinfo_5 @ 0x4f46940] config out time_base: 0/0, frame_rate: 0/0
 Output #0, null, to 'pipe:':
   Metadata:
     encoder         : Lavf58.13.100
     Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3],
 q=2-31, 200 kb/s, 15 fps, 15 tbn, 15 tbc (default)
     Metadata:
       encoder         : Lavc58.19.100 wrapped_avframe
 [Parsed_showinfo_5 @ 0x4f46940] n:   0 pts:      0 pts_time:0       pos:
 -1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:79FA842D
 plane_checksum:[79FA842D] mean:[1] stdev:[0.0]
 [Parsed_showinfo_5 @ 0x4f46940] n:   1 pts:      1 pts_time:0.0666667 pos:
 -1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:F3F40869
 plane_checksum:[F3F40869] mean:[2] stdev:[0.0]
 [Parsed_showinfo_5 @ 0x4f46940] n:   2 pts:      2 pts_time:0.133333 pos:
 -1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:E7F710D2
 plane_checksum:[E7F710D2] mean:[4] stdev:[0.0]
 [Parsed_showinfo_5 @ 0x4f46940] n:   3 pts:      3 pts_time:0.2     pos:
 -1 fmt:rgb24 sar:1/1 s:320x240 i:P iskey:1 type:I checksum:56039D68
 plane_checksum:[56039D68] mean:[7] stdev:[0.0]
 frame=    3 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.20 bitrate=N/A
 speed=8.27x
 video:2kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
 muxing overhead: unknown
 }}}

 Frame number (from 0) is the mean:[x] value. With round=near, it's
 [1,2,4,7]. With round=up and round=inf, it's [0,1,4,6]. With round=zero
 and round=down, it's [1,4,5,7]. With start_time=1 and '-t 1.2', it's
 [31,34,35,37]; however, with 'start_time=0.11:round=near' and '-t 0.31',
 it's [4,6,8,10].

 This report would be much better if I offered a patch, but I don't have
 the expertise.

 When framerate is halved, the fps filter uses an interval or point that's
 midway between the start point and the next input frame. I don't know how
 it's done mathematically or in the code, but this is how it works. Due to
 timebase rounding, sometimes the next frame is closer, and sometimes the
 previous one is.

 With input framerate = output framerate, the fps filter will still
 duplicate and drop frames with some rounding methods; in this case,
 'fps=30:start_time=0:round=down' leads to [1,1,2,4,4,5,7].

 The frame duplication in the first case can be avoided by using a
 start_time offset that exceeds the variation in frames. But if YouTube can
 get this wrong, normal users can as well. At least one of the rounding
 methods should work without adjusting the start time, and it might as well
 be 'near'.

 Conceptually, the interval should be centered on the first frame, not on
 the average between first and last frames of the input interval. So for
 input=30fps and output=30fps, the interval is [-1/60,1/60]. Frame at 0 is
 closest. For output=15fps, interval is [-1/30,1/30]. Frame at 0 is still
 closest. Second interval for 15 fps is [1/30,3/30], and the frame at
 pts=67:pts_time=0.067 is closest. There's one very obvious candidate
 instead of two very similar candidate frames.

 This can still lead to problems if the input timebase isn't divisible by
 framerate, and the offset gradually or suddenly jumps so that the decision
 point is equidistant from two frames. Could happen with variable framerate
 video that has been edited; when concatenating two videos with ffmpeg,
 which leads to odd offsets due to audio having a different length than
 video, like the second clip being 0.01 sec early or late; or when
 converting ~59.97 fps video to 30 fps.

 The muxers in ffmpeg will add or drop a frame based on an acceptable
 offset, like 0.5 or 1.5 times the distance between frames based on
 framerate from -r [rate]. The fps filter could try to select the next
 input frame to use for output based on the previous one used, by
 attempting to moderate the high-frequency variations introduced by
 timebase rounding, but this would be a more extensive patch and just
 making round=near work for halving framerate would a good fix by itself.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/6006#comment:4>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list