Explanation of how -r affects frame rate of filtergraph
Hello, I've seen some odd behavior with using -r that I cannot see documented. Twos input are a png images (just pure white in this example), another is a 30fps mp4. The output is set to 30fps using the "-r" option. ffmpeg -y -loop 1 -i background.png -i source.numbered.mp4 -loop 1 -i background.png -filter_complex_script filtergraph -r 30.0 -colorspace bt709 -color_trc bt709 -color_primaries bt709 -color_range tv r_before_output.mp4 This results in a 30fps video but frames are dropped from the video source, resulting in duplicates in the final output. A "fix" seems to be to specify the frame rate of the first image input, and not of the output at all. Like so: ffmpeg -y -loop 1 -r 30 -i background.png -i source.numbered.mp4 -loop 1 -i background.png -filter_complex_script filtergraph -colorspace bt709 -color_trc bt709 -color_primaries bt709 -color_range tv r_before_first_input.mp4 This also results in a 30fps output with all frames present but seems counter intuitive to put the frame rate on an image input. I wasn't able to find documentation on this behavior. Is someone able to explain how this is working internally? And whether or not this "fix" is actually reliable or just a lucky coincidence? Thanks, Peter Full console output from the commands listed above: $ cat filtergraph [1]drawtext=fontfile=Arial.ttf: text='start %{frame_num}': start_number=1: x=10: y=h-(2*lh): fontcolor=black: fontsize=20: box=1: boxcolor=white: boxborderw=5[trimmed1]; [trimmed1]scale=out_color_matrix=bt709[vcolored1]; [vcolored1]drawtext=fontfile=Arial.ttf: text='beforeloop %{frame_num}': start_number=1: x=400: y=h-(2*lh): fontcolor=black: fontsize=20: box=1: boxcolor=white: boxborderw=5[vlooped1]; [vlooped1]format=pix_fmts=rgba,pad=w=max(1280.000000\,0.000000+iw)-min(0\,0.000000):h=max(720.000000\,0.000000+ih)-min(0\,0.000000):x=max(0\,0.000000):y=max(0\,0.000000):color=black@0.0 ,crop=w=1280:h=720:x=0+abs(min(0\,0.000000)):y=0+abs(min(0\,0.000000)):w=1280:h=720:x=0+abs(min(0\,0.000000)):y=0+abs(min(0\,0.000000))[tx1]; [tx1]drawtext=fontfile=Arial.ttf: text='before blend %{frame_num}': start_number=1: x=700: y=h-(2*lh): fontcolor=black: fontsize=20: box=1: boxcolor=white: boxborderw=5[txx1]; [txx1][2]blend=c3_mode=multiply:shortest=1[o1]; [0][o1]overlay=x=0.0:y=0.0:shortest=1[stacked]; [stacked]drawtext=fontfile=Arial.ttf: text='end %{frame_num}': start_number=1: x=1000: y=h-(2*lh): fontcolor=black: fontsize=20: box=1: boxcolor=white: boxborderw=5[cropped]; [cropped]null $ ffmpeg -y -loop 1 -i background.png -i source.numbered.mp4 -loop 1 -i background.png -filter_complex_script filtergraph -r 30.0 -colorspace bt709 -color_trc bt709 -color_primaries bt709 -color_range tv r_before_output.mp4 ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers built with Apple clang version 11.0.0 (clang-1100.0.33.17) configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, png_pipe, from 'background.png': Duration: N/A, bitrate: N/A Stream #0:0: Video: png, monob(pc), 1280x720, 25 fps, 25 tbr, 25 tbn, 25 tbc Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'source.numbered.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.29.100 Duration: 00:00:05.07, start: 0.000000, bitrate: 4043 kb/s Stream #1:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720, 4039 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default) Metadata: handler_name : L-SMASH Video Handler Input #2, png_pipe, from 'background.png': Duration: N/A, bitrate: N/A Stream #2:0: Video: png, monob(pc), 1280x720, 25 fps, 25 tbr, 25 tbn, 25 tbc [Parsed_drawtext_0 @ 0x7fdc88500000] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_2 @ 0x7fdca850bc80] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_6 @ 0x7fdc8840cb00] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_9 @ 0x7fdca8425e40] Using "/Library/Fonts/Verdana.ttf" Stream mapping: Stream #0:0 (png) -> overlay:main Stream #1:0 (h264) -> drawtext Stream #2:0 (png) -> blend:bottom null -> Stream #0:0 (libx264) Press [q] to stop, [?] for help [Parsed_drawtext_0 @ 0x7fdc58400d00] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_2 @ 0x7fdc89805400] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_6 @ 0x7fdca8711e80] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_9 @ 0x7fdca8588680] Using "/Library/Fonts/Verdana.ttf" [swscaler @ 0x7fdcb0440000] deprecated pixel format used, make sure you did set range correctly [libx264 @ 0x7fdca900ea00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fdca900ea00] profile High, level 3.1 [libx264 @ 0x7fdca900ea00] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'r_before_output.mp4': Metadata: encoder : Lavf58.29.100 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1280x720, q=-1--1, 30 fps, 15360 tbn, 30 tbc (default) Metadata: encoder : Lavc58.54.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1 frame= 151 fps= 72 q=-1.0 Lsize= 2149kB time=00:00:04.93 bitrate=3568.7kbits/s dup=25 drop=0 speed=2.37x video:2147kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.122016% [libx264 @ 0x7fdca900ea00] frame I:1 Avg QP:25.04 size:162815 [libx264 @ 0x7fdca900ea00] frame P:50 Avg QP:25.26 size: 22169 [libx264 @ 0x7fdca900ea00] frame B:100 Avg QP:30.86 size: 9261 [libx264 @ 0x7fdca900ea00] consecutive B-frames: 0.7% 33.1% 0.0% 66.2% [libx264 @ 0x7fdca900ea00] mb I I16..4: 4.1% 41.4% 54.6% [libx264 @ 0x7fdca900ea00] mb P I16..4: 0.9% 14.9% 3.6% P16..4: 27.9% 6.1% 4.5% 0.0% 0.0% skip:42.2% [libx264 @ 0x7fdca900ea00] mb B I16..4: 0.5% 7.0% 1.4% B16..8: 15.5% 4.1% 1.5% direct: 1.5% skip:68.5% L0:63.6% L1:32.2% BI: 4.1% [libx264 @ 0x7fdca900ea00] 8x8 transform intra:75.8% inter:68.0% [libx264 @ 0x7fdca900ea00] coded y,uvDC,uvAC intra: 78.1% 84.0% 33.9% inter: 10.5% 15.9% 1.4% [libx264 @ 0x7fdca900ea00] i16 v,h,dc,p: 22% 31% 4% 43% [libx264 @ 0x7fdca900ea00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 13% 10% 8% 7% 10% 7% 12% 11% [libx264 @ 0x7fdca900ea00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 18% 11% 7% 6% 8% 6% 8% 6% [libx264 @ 0x7fdca900ea00] i8c dc,h,v,p: 42% 20% 26% 13% [libx264 @ 0x7fdca900ea00] Weighted P-Frames: Y:2.0% UV:0.0% [libx264 @ 0x7fdca900ea00] ref P L0: 52.5% 19.7% 16.6% 10.9% 0.3% [libx264 @ 0x7fdca900ea00] ref B L0: 78.7% 18.5% 2.8% [libx264 @ 0x7fdca900ea00] ref B L1: 99.8% 0.2% [libx264 @ 0x7fdca900ea00] kb/s:3492.52 $ ffmpeg -y -loop 1 -r 30 -i background.png -i source.numbered.mp4 -loop 1 -i background.png -filter_complex_script filtergraph -colorspace bt709 -color_trc bt709 -color_primaries bt709 -color_range tv r_before_first_input.mp4 ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers built with Apple clang version 11.0.0 (clang-1100.0.33.17) configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 Input #0, png_pipe, from 'background.png': Duration: N/A, bitrate: N/A Stream #0:0: Video: png, monob(pc), 1280x720, 25 fps, 25 tbr, 25 tbn, 25 tbc Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'source.numbered.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.29.100 Duration: 00:00:05.07, start: 0.000000, bitrate: 4043 kb/s Stream #1:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720, 4039 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default) Metadata: handler_name : L-SMASH Video Handler Input #2, png_pipe, from 'background.png': Duration: N/A, bitrate: N/A Stream #2:0: Video: png, monob(pc), 1280x720, 25 fps, 25 tbr, 25 tbn, 25 tbc [Parsed_drawtext_0 @ 0x7f9e48d24380] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_2 @ 0x7f9e48d27480] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_6 @ 0x7f9e28f01f80] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_9 @ 0x7f9e48d26380] Using "/Library/Fonts/Verdana.ttf" Stream mapping: Stream #0:0 (png) -> overlay:main Stream #1:0 (h264) -> drawtext Stream #2:0 (png) -> blend:bottom null -> Stream #0:0 (libx264) Press [q] to stop, [?] for help [Parsed_drawtext_0 @ 0x7f9e2b007080] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_2 @ 0x7f9e2b027a40] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_6 @ 0x7f9e2b012f40] Using "/Library/Fonts/Verdana.ttf" [Parsed_drawtext_9 @ 0x7f9e2b00df40] Using "/Library/Fonts/Verdana.ttf" [swscaler @ 0x7f9e40798000] deprecated pixel format used, make sure you did set range correctly [libx264 @ 0x7f9e4900d800] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7f9e4900d800] profile High, level 3.1 [libx264 @ 0x7f9e4900d800] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'r_before_first_input.mp4': Metadata: encoder : Lavf58.29.100 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1280x720, q=-1--1, 30 fps, 15360 tbn, 30 tbc (default) Metadata: encoder : Lavc58.54.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1 frame= 152 fps= 70 q=-1.0 Lsize= 2369kB time=00:00:04.96 bitrate=3907.4kbits/s speed=2.29x video:2366kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.110847% [libx264 @ 0x7f9e4900d800] frame I:1 Avg QP:24.90 size:164402 [libx264 @ 0x7f9e4900d800] frame P:38 Avg QP:25.46 size: 23113 [libx264 @ 0x7f9e4900d800] frame B:113 Avg QP:31.00 size: 12210 [libx264 @ 0x7f9e4900d800] consecutive B-frames: 0.7% 0.0% 2.0% 97.4% [libx264 @ 0x7f9e4900d800] mb I I16..4: 3.6% 41.6% 54.8% [libx264 @ 0x7f9e4900d800] mb P I16..4: 1.1% 15.6% 3.4% P16..4: 30.0% 6.2% 4.6% 0.0% 0.0% skip:39.0% [libx264 @ 0x7f9e4900d800] mb B I16..4: 0.6% 9.0% 1.9% B16..8: 16.4% 5.6% 2.1% direct: 2.1% skip:62.4% L0:47.0% L1:48.1% BI: 4.9% [libx264 @ 0x7f9e4900d800] 8x8 transform intra:76.4% inter:69.5% [libx264 @ 0x7f9e4900d800] coded y,uvDC,uvAC intra: 76.7% 82.6% 33.1% inter: 12.2% 17.1% 1.5% [libx264 @ 0x7f9e4900d800] i16 v,h,dc,p: 24% 29% 4% 42% [libx264 @ 0x7f9e4900d800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 13% 10% 8% 7% 10% 7% 12% 11% [libx264 @ 0x7f9e4900d800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 18% 12% 7% 6% 8% 6% 8% 6% [libx264 @ 0x7f9e4900d800] i8c dc,h,v,p: 42% 20% 25% 13% [libx264 @ 0x7f9e4900d800] Weighted P-Frames: Y:2.6% UV:2.6% [libx264 @ 0x7f9e4900d800] ref P L0: 47.7% 22.4% 19.5% 10.1% 0.4% [libx264 @ 0x7f9e4900d800] ref B L0: 76.0% 17.6% 6.4% [libx264 @ 0x7f9e4900d800] ref B L1: 89.8% 10.2% [libx264 @ 0x7f9e4900d800] kb/s:3824.97
On Wed, Feb 26, 2020, at 4:02 PM, Peter wrote:
This also results in a 30fps output with all frames present but seems counter intuitive to put the frame rate on an image input. I wasn't able to find documentation on this behavior. Is someone able to explain how this is working internally? And whether or not this "fix" is actually reliable or just a lucky coincidence?
Default frame rate for the image demuxer is 25 unless you add the -framerate input option. https://ffmpeg.org/ffmpeg-formats.html#image2-1
Peter (12020-02-27):
This also results in a 30fps output with all frames present but seems counter intuitive to put the frame rate on an image input.
FFmpeg works with videos. You are feeding it an image, it's considered as a video, it needs a frame rate. If you don't specify it, you get a default value. Relying on default value is rarely a good idea.
Is someone able to explain how this is working internally?
All frames have a timestamp. If there is a regular frame rate, it is only the inverse of the constant time difference between frames.
And whether or not this "fix" is actually reliable or just a lucky coincidence?
Yes, except the option is really named framerate. Regards, -- Nicolas George
participants (3)
-
Lou Logan -
Nicolas George -
Peter