gif overlay miscalculating x and y, extra frame flash
Hi, I continued building and rendering after asking some questions last week. After 6 days x10 hours per day of (render,test,measure,edit,repeat) I finally was able to publish my video. In doing so I made it easy to duplicate the bugs I have been seeing. I have put the 7 files together in one 56Mb package for you to download at https://www.seahorsecorral.org/videos/tests/ffmpeg-test-gif.zip The issue is with -i gif and overlay=x=(calculations):y=(calculations):between 1. between calculates n differently than the 30fps video, 2. the last calculation for x and y are held over in memory for the next overlay, resulting in offsets and drift over time. 3. occasionally one frame of the gif appears again, outside the range given in between, in addition to the frames shown at the correct time (ignoring the 33.333/30 drift). 4. occasionally one frame inside that range specified in between, is left out or skipped. I made an extensive list of when some or all of the above are seen and repeatable. It is in the comments at the end of the .sh file. The test1.sh version only loaded one video and one gif file, and thus only saw problem #1. By adding a second gif with different color, (this test2.sh) then the flashes of the single frame can be identified to it's parent. The script generates the test-run "2n" where an orange flash is at frame 408 and a red flash at frame 571. If my comments are unclear, just ask. Procedure to duplicate: $ ./build-test2.sh p for the perfect png version, which duplicates everything except the input file type, and then for the gif version, here is the full output as always requested: $ ./build-test2.sh ffmpeg version N-96573-g73aab08d9d Copyright (c) 2000-2020 the FFmpeg developers built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516 configuration: --prefix=/usr/local --enable-gpl --enable-libmp3lame --enable-libvorbis --enable-libx264 --enable-libopenjpeg --enable-libfreetype --disable-doc --disable-htmlpages --disable-podpages --enable-shared --enable-libvpx --extra-cflags=-I/usr/include --extra-ldflags=-L/usr/lib/i386-linux-gnu --enable-libass --enable-libtesseract --extra-cflags=-msse2 --enable-openssl --enable-nonfree libavutil 56. 38.100 / 56. 36.101 libavcodec 58. 67.100 / 58. 64.101 libavformat 58. 37.100 / 58. 35.101 libavdevice 58. 9.103 / 58. 9.101 libavfilter 7. 72.100 / 7. 68.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x180e140] st: 0 edit list: 1 Missing key frame while searching for timestamp: 2000 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x180e140] st: 0 edit list 1 Cannot find an index entry before timestamp: 2000. Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '20190628_163959_driving-WCurlewLkRd-S_30sec_FHDw.MOV': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2019-06-28T16:39:59.000000Z playback_requirements: QuickTime 6.0 or greater playback_requirements-eng: QuickTime 6.0 or greater Duration: 00:00:29.50, start: 0.000000, bitrate: 15212 kb/s Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 15078 kb/s, 30 fps, 30 tbr, 60k tbn, 120k tbc (default) Metadata: creation_time : 2019-06-28T16:39:59.000000Z handler_name : Video Handler encoder : iCatch AVCC Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default) Metadata: creation_time : 2019-06-28T16:39:59.000000Z handler_name : Audio Handler Input #1, gif, from 'redwave-500x400.gif': Duration: 00:00:01.35, start: 0.000000, bitrate: 2635 kb/s Stream #1:0: Video: gif, bgra, 500x400, 34 fps, 100 tbr, 100 tbn, 100 tbc Input #2, gif, from 'orangewave-500x400.gif': Duration: 00:00:01.35, start: 0.000000, bitrate: 2635 kb/s Stream #2:0: Video: gif, bgra, 500x400, 34 fps, 100 tbr, 100 tbn, 100 tbc Stream mapping: Stream #0:0 (h264) -> scale (graph 0) Stream #1:0 (gif) -> split (graph 0) Stream #2:0 (gif) -> overlay:overlay (graph 0) drawtext (graph 0) -> Stream #0:0 (libx264) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) Press [q] to stop, [?] for help [libx264 @ 0x18d2bc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX LZCNT BMI1 SlowPshufb [libx264 @ 0x18d2bc0] profile High, level 3.1 [libx264 @ 0x18d2bc0] 264 - core 148 r2748 97eaef2 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=2 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'test2-out-gif-720e.mp4': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt playback_requirements-eng: QuickTime 6.0 or greater playback_requirements: QuickTime 6.0 or greater encoder : Lavf58.35.101 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720, q=-1--1, 30 fps, 15360 tbn, 30 tbc (default) Metadata: encoder : Lavc58.67.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Metadata: creation_time : 2019-06-28T16:39:59.000000Z handler_name : Audio Handler encoder : Lavc58.67.100 aac [mp4 @ 0x18d2100] Starting second pass: moving the moov atom to the beginning of the file0.181x frame= 840 fps=5.3 q=-1.0 Lsize= 42273kB time=00:00:28.01 bitrate=12363.2kbits/s speed=0.177x video:41964kB audio:280kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.070744% [libx264 @ 0x18d2bc0] frame I:56 Avg QP:17.71 size:150429 [libx264 @ 0x18d2bc0] frame P:280 Avg QP:20.84 size: 63961 [libx264 @ 0x18d2bc0] frame B:504 Avg QP:23.72 size: 33010 [libx264 @ 0x18d2bc0] consecutive B-frames: 6.7% 13.3% 80.0% [libx264 @ 0x18d2bc0] mb I I16..4: 5.9% 56.3% 37.9% [libx264 @ 0x18d2bc0] mb P I16..4: 0.9% 13.0% 6.1% P16..4: 32.6% 22.7% 15.8% 0.0% 0.0% skip: 9.0% [libx264 @ 0x18d2bc0] mb B I16..4: 0.1% 2.1% 1.9% B16..8: 39.2% 14.6% 7.8% direct: 4.2% skip:30.0% L0:43.9% L1:36.4% BI:19.7% [libx264 @ 0x18d2bc0] 8x8 transform intra:59.4% inter:60.1% [libx264 @ 0x18d2bc0] coded y,uvDC,uvAC intra: 92.6% 93.1% 68.5% inter: 35.9% 38.0% 9.1% [libx264 @ 0x18d2bc0] i16 v,h,dc,p: 6% 33% 4% 56% [libx264 @ 0x18d2bc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 26% 13% 6% 8% 7% 10% 7% 14% [libx264 @ 0x18d2bc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 28% 10% 7% 9% 7% 11% 6% 11% [libx264 @ 0x18d2bc0] i8c dc,h,v,p: 47% 31% 12% 9% [libx264 @ 0x18d2bc0] Weighted P-Frames: Y:6.1% UV:2.1% [libx264 @ 0x18d2bc0] ref P L0: 62.6% 14.8% 16.1% 6.2% 0.2% [libx264 @ 0x18d2bc0] ref B L0: 80.8% 16.5% 2.7% [libx264 @ 0x18d2bc0] kb/s:12277.17 [aac @ 0x18d0f40] Qavg: 118.000 Stewart
Am 09.02.2020 um 23:51 schrieb S Andreason:
Hi, I continued building and rendering after asking some questions last week. After 6 days x10 hours per day of (render,test,measure,edit,repeat) I finally was able to publish my video. In doing so I made it easy to duplicate the bugs I have been seeing.
I have put the 7 files together in one 56Mb package for you to download at https://www.seahorsecorral.org/videos/tests/ffmpeg-test-gif.zip
The issue is with -i gif and overlay=x=(calculations):y=(calculations):between 1. between calculates n differently than the 30fps video, 2. the last calculation for x and y are held over in memory for the next overlay, resulting in offsets and drift over time.
I have a similar problem, not sure if it's the same as yours, but it's easier to reproduce. A small black rectangle is overlaid over a video, and its position is shifted to the right by one pixel per second. Theoretically we should see 9 steps in the 10s video. But I see only 4 steps, in 2s intervals. ffmpeg -f lavfi -i testsrc2=size=vga -f lavfi -i color=black:size=20x40 -lavfi overlay=x=400+t:y=200 -c:v mpeg4 -q:v 0 -y -t 10 out.mp4 The console output is below. Michael C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -f lavfi -i testsrc2=size=vga -f lavfi - i color=black:size=20x40 -lavfi overlay=x=400+t:y=200 -c:v mpeg4 -q:v 0 -y -t 10 out.mp4 ffmpeg version git-2020-02-06-343ccfc Copyright (c) 2000-2020 the FFmpeg develop ers built with gcc 9.2.1 (GCC) 20200122 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfi g --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libb luray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enab le-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --e nable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable -libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 - -enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enab le-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --en able-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcode c --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 - -enable-avisynth --enable-libopenmpt --enable-amf libavutil 56. 39.100 / 56. 39.100 libavcodec 58. 68.100 / 58. 68.100 libavformat 58. 38.100 / 58. 38.100 libavdevice 58. 9.103 / 58. 9.103 libavfilter 7. 74.100 / 7. 74.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 Input #0, lavfi, from 'testsrc2=size=vga': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc Input #1, lavfi, from 'color=black:size=20x40': Duration: N/A, start: 0.000000, bitrate: N/A Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 20x40 [SAR 1:1 DA R 1:2], 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 (rawvideo) -> overlay:main Stream #1:0 (rawvideo) -> overlay:overlay overlay -> Stream #0:0 (mpeg4) Press [q] to stop, [?] for help Output #0, mp4, to 'out.mp4': Metadata: encoder : Lavf58.38.100 Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc (default) Metadata: encoder : Lavc58.68.100 mpeg4 Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A frame= 250 fps=0.0 q=0.0 Lsize= 6089kB time=00:00:09.96 bitrate=5007.8kbits/ s speed=30.5x video:6087kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.032490%
On 2/10/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 09.02.2020 um 23:51 schrieb S Andreason:
Hi, I continued building and rendering after asking some questions last week. After 6 days x10 hours per day of (render,test,measure,edit,repeat) I finally was able to publish my video. In doing so I made it easy to duplicate the bugs I have been seeing.
I have put the 7 files together in one 56Mb package for you to download at https://www.seahorsecorral.org/videos/tests/ffmpeg-test-gif.zip
The issue is with -i gif and overlay=x=(calculations):y=(calculations):between 1. between calculates n differently than the 30fps video, 2. the last calculation for x and y are held over in memory for the next overlay, resulting in offsets and drift over time.
I have a similar problem, not sure if it's the same as yours, but it's easier to reproduce. A small black rectangle is overlaid over a video, and its position is shifted to the right by one pixel per second. Theoretically we should see 9 steps in the 10s video. But I see only 4 steps, in 2s intervals.
ffmpeg -f lavfi -i testsrc2=size=vga -f lavfi -i color=black:size=20x40 -lavfi overlay=x=400+t:y=200 -c:v mpeg4 -q:v 0 -y -t 10 out.mp4
You reasoning is completely invalid. And ffmpeg output is correct. Next time read more documentation with understanding and learn how to use overlay format option.
The console output is below.
Michael
C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -f lavfi -i testsrc2=size=vga -f lavfi - i color=black:size=20x40 -lavfi overlay=x=400+t:y=200 -c:v mpeg4 -q:v 0 -y -t 10 out.mp4 ffmpeg version git-2020-02-06-343ccfc Copyright (c) 2000-2020 the FFmpeg develop ers built with gcc 9.2.1 (GCC) 20200122 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfi g --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libb luray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enab le-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --e nable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable -libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 - -enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enab le-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --en able-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcode c --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 - -enable-avisynth --enable-libopenmpt --enable-amf libavutil 56. 39.100 / 56. 39.100 libavcodec 58. 68.100 / 58. 68.100 libavformat 58. 38.100 / 58. 38.100 libavdevice 58. 9.103 / 58. 9.103 libavfilter 7. 74.100 / 7. 74.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 Input #0, lavfi, from 'testsrc2=size=vga': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc Input #1, lavfi, from 'color=black:size=20x40': Duration: N/A, start: 0.000000, bitrate: N/A Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 20x40 [SAR 1:1 DA R 1:2], 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 (rawvideo) -> overlay:main Stream #1:0 (rawvideo) -> overlay:overlay overlay -> Stream #0:0 (mpeg4) Press [q] to stop, [?] for help Output #0, mp4, to 'out.mp4': Metadata: encoder : Lavf58.38.100 Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc (default) Metadata: encoder : Lavc58.68.100 mpeg4 Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A frame= 250 fps=0.0 q=0.0 Lsize= 6089kB time=00:00:09.96 bitrate=5007.8kbits/ s speed=30.5x video:6087kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.032490%
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
Am 10.02.2020 um 09:35 schrieb Paul B Mahol:
On 2/10/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 09.02.2020 um 23:51 schrieb S Andreason:
Hi, I continued building and rendering after asking some questions last week. After 6 days x10 hours per day of (render,test,measure,edit,repeat) I finally was able to publish my video. In doing so I made it easy to duplicate the bugs I have been seeing.
I have put the 7 files together in one 56Mb package for you to download at https://www.seahorsecorral.org/videos/tests/ffmpeg-test-gif.zip
The issue is with -i gif and overlay=x=(calculations):y=(calculations):between 1. between calculates n differently than the 30fps video, 2. the last calculation for x and y are held over in memory for the next overlay, resulting in offsets and drift over time. I have a similar problem, not sure if it's the same as yours, but it's easier to reproduce. A small black rectangle is overlaid over a video, and its position is shifted to the right by one pixel per second. Theoretically we should see 9 steps in the 10s video. But I see only 4 steps, in 2s intervals.
ffmpeg -f lavfi -i testsrc2=size=vga -f lavfi -i color=black:size=20x40 -lavfi overlay=x=400+t:y=200 -c:v mpeg4 -q:v 0 -y -t 10 out.mp4
You reasoning is completely invalid. And ffmpeg output is correct. Next time read more documentation with understanding and learn how to use overlay format option.
When I add :format=rgb then all steps are indeed visible. I don't understand why it doesn't work with the default yuv420 format. Doesn't yuv420 mean "full resolution for luminance, half resolution for color"? The black rectangle has a different luminance than the background. Can you please explain why half of the steps are invisible? Nevertheless I have the same problem as the original poster, also when I add :format=rgb. Something is wrong with the x,y calculations in the overlay filter. In my example I'm using sendcmd to set x and y. The coordinates are sometimes off by a few pixels. At the moment my command line is too complicated to show here. Give me a little bit of time to find an example that's easier to reproduce. Michael
On 2/10/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 10.02.2020 um 09:35 schrieb Paul B Mahol:
On 2/10/20, Michael Koch <astroelectronic@t-online.de> wrote:
Am 09.02.2020 um 23:51 schrieb S Andreason:
Hi, I continued building and rendering after asking some questions last week. After 6 days x10 hours per day of (render,test,measure,edit,repeat) I finally was able to publish my video. In doing so I made it easy to duplicate the bugs I have been seeing.
I have put the 7 files together in one 56Mb package for you to download at https://www.seahorsecorral.org/videos/tests/ffmpeg-test-gif.zip
The issue is with -i gif and overlay=x=(calculations):y=(calculations):between 1. between calculates n differently than the 30fps video, 2. the last calculation for x and y are held over in memory for the next overlay, resulting in offsets and drift over time. I have a similar problem, not sure if it's the same as yours, but it's easier to reproduce. A small black rectangle is overlaid over a video, and its position is shifted to the right by one pixel per second. Theoretically we should see 9 steps in the 10s video. But I see only 4 steps, in 2s intervals.
ffmpeg -f lavfi -i testsrc2=size=vga -f lavfi -i color=black:size=20x40 -lavfi overlay=x=400+t:y=200 -c:v mpeg4 -q:v 0 -y -t 10 out.mp4
You reasoning is completely invalid. And ffmpeg output is correct. Next time read more documentation with understanding and learn how to use overlay format option.
When I add :format=rgb then all steps are indeed visible. I don't understand why it doesn't work with the default yuv420 format. Doesn't yuv420 mean "full resolution for luminance, half resolution for color"? The black rectangle has a different luminance than the background. Can you please explain why half of the steps are invisible?
Because yuv420 uses half resolution for color, and even no color means it needs to store some value in frame, 128 in this case.
Nevertheless I have the same problem as the original poster, also when I add :format=rgb. Something is wrong with the x,y calculations in the overlay filter. In my example I'm using sendcmd to set x and y. The coordinates are sometimes off by a few pixels. At the moment my command line is too complicated to show here. Give me a little bit of time to find an example that's easier to reproduce.
No, as already said if timebases and fps do not match you can not get results you expect. Use fps filter somehow...
Michael
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
Paul,
Nevertheless I have the same problem as the original poster, also when I add :format=rgb. Something is wrong with the x,y calculations in the overlay filter. In my example I'm using sendcmd to set x and y. The coordinates are sometimes off by a few pixels. At the moment my command line is too complicated to show here. Give me a little bit of time to find an example that's easier to reproduce. No, as already said if timebases and fps do not match you can not get results you expect. Use fps filter somehow...
Here is a new example for unexpected behaviour of the overlay filter: c:\ffmpeg\ffmpeg -f lavfi -i testsrc2=size=hd1080:duration=10 -f lavfi -i color=black:size=50x50:duration=7 -lavfi "[1]sendcmd=f=test.cmd[2];[0][2]overlay=x=0:y=0:format=rgb" -t 10 -y out.mp4 The coordinates come from this test.cmd file: 0.0-1.0 overlay x 'lerp(1735-50,1723-50,t)', overlay y 'lerp( 527-50, 529-50,t)'; 1.0-2.0 overlay x 'lerp(1723-50,1605-50,t-1)', overlay y 'lerp( 529-50, 526-50,t-1)'; 2.0-3.0 overlay x 'lerp(1605-50,1440-50,t-2)', overlay y 'lerp( 526-50, 526-50,t-2)'; 3.0-4.0 overlay x 'lerp(1440-50,1266-50,t-3)', overlay y 'lerp( 526-50, 523-50,t-3)'; 4.0-5.0 overlay x 'lerp(1266-50,1113-50,t-4)', overlay y 'lerp( 523-50, 523-50,t-4)'; 5.0-6.0 overlay x 'lerp(1113-50,1007-50,t-5)', overlay y 'lerp( 523-50, 522-50,t-5)'; 6.0-7.0 overlay x 'lerp(1007-50, 966-50,t-6)', overlay y 'lerp( 522-50, 522-50,t-6)'; 7.0-8.0 overlay x 'lerp( 966-50, 964-50,t-7)', overlay y 'lerp( 522-50, 522-50,t-7)'; 8.0-9.0 overlay x 'lerp( 964-50, 964-50,t-8)', overlay y 'lerp( 522-50, 522-50,t-8)'; 9.0-10.0 overlay x 'lerp( 964-50, 964-50,t-9)', overlay y 'lerp( 522-50, 522-50,t-9)'; Please note that all movements end at 8s. No movement is specified from 8s to 10s. In the command line I have two different durations, 10s for the main video and 7s for the overlay. Expected behaviour would be that after 7s the last frame is used and overlaid at the correct position. But it seems that sendcmd gets the wrong time, so that there is movement visible until the end at 10s. Everything is ok when you change the overlay duration to 10s. Michael F:\Test_Wormhole_2020>c:\ffmpeg\ffmpeg -f lavfi -i testsrc2=size=hd1080:duration =10 -f lavfi -i color=black:size=50x50:duration=7 -lavfi "[1]sendcmd=f=test.cmd[ 2];[0][2]overlay=x=0:y=0:format=rgb" -t 10 -y out.mp4 ffmpeg version git-2020-02-06-343ccfc Copyright (c) 2000-2020 the FFmpeg develop ers built with gcc 9.2.1 (GCC) 20200122 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfi g --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libb luray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enab le-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --e nable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable -libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 - -enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enab le-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --en able-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcode c --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 - -enable-avisynth --enable-libopenmpt --enable-amf libavutil 56. 39.100 / 56. 39.100 libavcodec 58. 68.100 / 58. 68.100 libavformat 58. 38.100 / 58. 38.100 libavdevice 58. 9.103 / 58. 9.103 libavfilter 7. 74.100 / 7. 74.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 Input #0, lavfi, from 'testsrc2=size=hd1080:duration=10': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080 [SAR 1: 1 DAR 16:9], 25 tbr, 25 tbn, 25 tbc Input #1, lavfi, from 'color=black:size=50x50:duration=7': Duration: N/A, start: 0.000000, bitrate: N/A Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 50x50 [SAR 1:1 DA R 1:1], 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 (rawvideo) -> overlay:main Stream #1:0 (rawvideo) -> sendcmd overlay -> Stream #0:0 (libx264) Press [q] to stop, [?] for help [swscaler @ 00000000036417c0] No accelerated colorspace conversion found from yu v420p to argb. [libx264 @ 00000000035a0880] using SAR=1/1 [libx264 @ 00000000035a0880] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 00000000035a0880] profile High 4:4:4 Predictive, level 4.0, 4:4:4, 8- bit [libx264 @ 00000000035a0880] 264 - core 159 - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1 :0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_rang e=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_ offset=4 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlac ed=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias= 0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 i ntra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax =69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'out.mp4': Metadata: encoder : Lavf58.38.100 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p, 1920x1080 [ SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default) Metadata: encoder : Lavc58.68.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A frame= 25 fps=nan q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= frame= 46 fps=nan q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= frame= 58 fps=nan q=28.0 size= 0kB time=00:00:00.20 bitrate= 1.9kbits/ frame= 70 fps=nan q=28.0 size= 512kB time=00:00:00.68 bitrate=6168.0kbits/ frame= 82 fps=nan q=28.0 size= 768kB time=00:00:01.16 bitrate=5423.6kbits/ frame= 98 fps= 29 q=28.0 size= 1024kB time=00:00:01.80 bitrate=4660.3kbits/ frame= 112 fps=nan q=28.0 size= 1536kB time=00:00:02.36 bitrate=5331.7kbits/ frame= 127 fps=nan q=28.0 size= 2048kB time=00:00:02.96 bitrate=5668.0kbits/ frame= 138 fps=nan q=28.0 size= 2304kB time=00:00:03.40 bitrate=5551.3kbits/ frame= 151 fps=nan q=28.0 size= 2560kB time=00:00:03.92 bitrate=5349.9kbits/ frame= 164 fps=nan q=28.0 size= 2816kB time=00:00:04.44 bitrate=5195.6kbits/ [Parsed_color_0 @ 000000000062ddc0] EOF timestamp not reliable frame= 179 fps= 27 q=28.0 size= 3328kB time=00:00:05.04 bitrate=5409.3kbits/ frame= 192 fps= 27 q=28.0 size= 3584kB time=00:00:05.56 bitrate=5280.6kbits/ frame= 207 fps= 27 q=28.0 size= 4096kB time=00:00:06.16 bitrate=5447.1kbits/ frame= 219 fps= 27 q=28.0 size= 4352kB time=00:00:06.64 bitrate=5369.2kbits/ frame= 232 fps= 27 q=28.0 size= 4608kB time=00:00:07.16 bitrate=5272.2kbits/ [Parsed_testsrc2_0 @ 00000000005a2ac0] EOF timestamp not reliable frame= 247 fps= 27 q=28.0 size= 5120kB time=00:00:07.76 bitrate=5405.0kbits/ frame= 250 fps= 24 q=-1.0 Lsize= 6573kB time=00:00:09.88 bitrate=5449.7kbits /s speed=0.934x video:6569kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.050334% [libx264 @ 00000000035a0880] frame I:1 Avg QP:13.68 size: 37689 [libx264 @ 00000000035a0880] frame P:113 Avg QP:18.14 size: 30113 [libx264 @ 00000000035a0880] frame B:136 Avg QP:29.27 size: 24161 [libx264 @ 00000000035a0880] consecutive B-frames: 18.4% 12.0% 45.6% 24.0% [libx264 @ 00000000035a0880] mb I I16..4: 40.9% 52.9% 6.2% [libx264 @ 00000000035a0880] mb P I16..4: 2.8% 2.2% 1.0% P16..4: 3.7% 3.1 % 1.7% 0.0% 0.0% skip:85.4% [libx264 @ 00000000035a0880] mb B I16..4: 0.2% 0.1% 1.0% B16..8: 7.6% 1.5 % 0.2% direct: 3.3% skip:86.0% L0:53.0% L1:38.9% BI: 8.1% [libx264 @ 00000000035a0880] 8x8 transform intra:33.5% inter:12.8% [libx264 @ 00000000035a0880] coded y,u,v intra: 22.3% 23.0% 22.6% inter: 3.3% 4. 1% 3.5% [libx264 @ 00000000035a0880] i16 v,h,dc,p: 88% 10% 1% 0% [libx264 @ 00000000035a0880] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 6% 9% 84% 0% 0% 0% 0% 0% 0% [libx264 @ 00000000035a0880] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 25% 29% 6% 3% 4% 4% 5% 4% [libx264 @ 00000000035a0880] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 00000000035a0880] ref P L0: 40.3% 8.6% 28.5% 22.6% [libx264 @ 00000000035a0880] ref B L0: 73.5% 21.2% 5.3% [libx264 @ 00000000035a0880] ref B L1: 96.8% 3.2% [libx264 @ 00000000035a0880] kb/s:5381.13
Am 10.02.2020 um 18:31 schrieb Michael Koch:
Paul,
Nevertheless I have the same problem as the original poster, also when I add :format=rgb. Something is wrong with the x,y calculations in the overlay filter. In my example I'm using sendcmd to set x and y. The coordinates are sometimes off by a few pixels. At the moment my command line is too complicated to show here. Give me a little bit of time to find an example that's easier to reproduce. No, as already said if timebases and fps do not match you can not get results you expect. Use fps filter somehow...
Here is a new example for unexpected behaviour of the overlay filter:
c:\ffmpeg\ffmpeg -f lavfi -i testsrc2=size=hd1080:duration=10 -f lavfi -i color=black:size=50x50:duration=7 -lavfi "[1]sendcmd=f=test.cmd[2];[0][2]overlay=x=0:y=0:format=rgb" -t 10 -y out.mp4
The coordinates come from this test.cmd file:
0.0-1.0 overlay x 'lerp(1735-50,1723-50,t)', overlay y 'lerp( 527-50, 529-50,t)'; 1.0-2.0 overlay x 'lerp(1723-50,1605-50,t-1)', overlay y 'lerp( 529-50, 526-50,t-1)'; 2.0-3.0 overlay x 'lerp(1605-50,1440-50,t-2)', overlay y 'lerp( 526-50, 526-50,t-2)'; 3.0-4.0 overlay x 'lerp(1440-50,1266-50,t-3)', overlay y 'lerp( 526-50, 523-50,t-3)'; 4.0-5.0 overlay x 'lerp(1266-50,1113-50,t-4)', overlay y 'lerp( 523-50, 523-50,t-4)'; 5.0-6.0 overlay x 'lerp(1113-50,1007-50,t-5)', overlay y 'lerp( 523-50, 522-50,t-5)'; 6.0-7.0 overlay x 'lerp(1007-50, 966-50,t-6)', overlay y 'lerp( 522-50, 522-50,t-6)'; 7.0-8.0 overlay x 'lerp( 966-50, 964-50,t-7)', overlay y 'lerp( 522-50, 522-50,t-7)'; 8.0-9.0 overlay x 'lerp( 964-50, 964-50,t-8)', overlay y 'lerp( 522-50, 522-50,t-8)'; 9.0-10.0 overlay x 'lerp( 964-50, 964-50,t-9)', overlay y 'lerp( 522-50, 522-50,t-9)';
Please note that all movements end at 8s. No movement is specified from 8s to 10s.
In the command line I have two different durations, 10s for the main video and 7s for the overlay. Expected behaviour would be that after 7s the last frame is used and overlaid at the correct position. But it seems that sendcmd gets the wrong time, so that there is movement visible until the end at 10s.
I just found my error myself. The sendcmd command is inserted at the wrong place in the filter chain. It must be inserted where the longer of the two inputs is. Michael
On 2/9/20, S Andreason <sandreas41@gmail.com> wrote:
Hi, I continued building and rendering after asking some questions last week. After 6 days x10 hours per day of (render,test,measure,edit,repeat) I finally was able to publish my video. In doing so I made it easy to duplicate the bugs I have been seeing.
I have put the 7 files together in one 56Mb package for you to download at https://www.seahorsecorral.org/videos/tests/ffmpeg-test-gif.zip
The issue is with -i gif and overlay=x=(calculations):y=(calculations):between 1. between calculates n differently than the 30fps video,
No, it is completely correct, you give it different timebase and fps video inputs. overlay video is 33.33 while main video is 30.00. If you want consistent results make videos same timebase and fps. The rest all depends on 1. point.
2. the last calculation for x and y are held over in memory for the next overlay, resulting in offsets and drift over time. 3. occasionally one frame of the gif appears again, outside the range given in between, in addition to the frames shown at the correct time (ignoring the 33.333/30 drift). 4. occasionally one frame inside that range specified in between, is left out or skipped.
I made an extensive list of when some or all of the above are seen and repeatable. It is in the comments at the end of the .sh file. The test1.sh version only loaded one video and one gif file, and thus only saw problem #1. By adding a second gif with different color, (this test2.sh) then the flashes of the single frame can be identified to it's parent. The script generates the test-run "2n" where an orange flash is at frame 408 and a red flash at frame 571. If my comments are unclear, just ask.
Procedure to duplicate: $ ./build-test2.sh p for the perfect png version, which duplicates everything except the input file type, and then for the gif version, here is the full output as always requested: $ ./build-test2.sh
ffmpeg version N-96573-g73aab08d9d Copyright (c) 2000-2020 the FFmpeg developers built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516 configuration: --prefix=/usr/local --enable-gpl --enable-libmp3lame --enable-libvorbis --enable-libx264 --enable-libopenjpeg --enable-libfreetype --disable-doc --disable-htmlpages --disable-podpages --enable-shared --enable-libvpx --extra-cflags=-I/usr/include --extra-ldflags=-L/usr/lib/i386-linux-gnu --enable-libass --enable-libtesseract --extra-cflags=-msse2 --enable-openssl --enable-nonfree libavutil 56. 38.100 / 56. 36.101 libavcodec 58. 67.100 / 58. 64.101 libavformat 58. 37.100 / 58. 35.101 libavdevice 58. 9.103 / 58. 9.101 libavfilter 7. 72.100 / 7. 68.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x180e140] st: 0 edit list: 1 Missing key frame while searching for timestamp: 2000 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x180e140] st: 0 edit list 1 Cannot find an index entry before timestamp: 2000. Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '20190628_163959_driving-WCurlewLkRd-S_30sec_FHDw.MOV': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt creation_time : 2019-06-28T16:39:59.000000Z playback_requirements: QuickTime 6.0 or greater playback_requirements-eng: QuickTime 6.0 or greater Duration: 00:00:29.50, start: 0.000000, bitrate: 15212 kb/s Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 15078 kb/s, 30 fps, 30 tbr, 60k tbn, 120k tbc (default) Metadata: creation_time : 2019-06-28T16:39:59.000000Z handler_name : Video Handler encoder : iCatch AVCC Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default) Metadata: creation_time : 2019-06-28T16:39:59.000000Z handler_name : Audio Handler Input #1, gif, from 'redwave-500x400.gif': Duration: 00:00:01.35, start: 0.000000, bitrate: 2635 kb/s Stream #1:0: Video: gif, bgra, 500x400, 34 fps, 100 tbr, 100 tbn, 100 tbc Input #2, gif, from 'orangewave-500x400.gif': Duration: 00:00:01.35, start: 0.000000, bitrate: 2635 kb/s Stream #2:0: Video: gif, bgra, 500x400, 34 fps, 100 tbr, 100 tbn, 100 tbc Stream mapping: Stream #0:0 (h264) -> scale (graph 0) Stream #1:0 (gif) -> split (graph 0) Stream #2:0 (gif) -> overlay:overlay (graph 0) drawtext (graph 0) -> Stream #0:0 (libx264) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) Press [q] to stop, [?] for help [libx264 @ 0x18d2bc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX LZCNT BMI1 SlowPshufb [libx264 @ 0x18d2bc0] profile High, level 3.1 [libx264 @ 0x18d2bc0] 264 - core 148 r2748 97eaef2 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=2 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'test2-out-gif-720e.mp4': Metadata: major_brand : qt minor_version : 0 compatible_brands: qt playback_requirements-eng: QuickTime 6.0 or greater playback_requirements: QuickTime 6.0 or greater encoder : Lavf58.35.101 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720, q=-1--1, 30 fps, 15360 tbn, 30 tbc (default) Metadata: encoder : Lavc58.67.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Metadata: creation_time : 2019-06-28T16:39:59.000000Z handler_name : Audio Handler encoder : Lavc58.67.100 aac [mp4 @ 0x18d2100] Starting second pass: moving the moov atom to the beginning of the file0.181x frame= 840 fps=5.3 q=-1.0 Lsize= 42273kB time=00:00:28.01 bitrate=12363.2kbits/s speed=0.177x video:41964kB audio:280kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.070744% [libx264 @ 0x18d2bc0] frame I:56 Avg QP:17.71 size:150429 [libx264 @ 0x18d2bc0] frame P:280 Avg QP:20.84 size: 63961 [libx264 @ 0x18d2bc0] frame B:504 Avg QP:23.72 size: 33010 [libx264 @ 0x18d2bc0] consecutive B-frames: 6.7% 13.3% 80.0% [libx264 @ 0x18d2bc0] mb I I16..4: 5.9% 56.3% 37.9% [libx264 @ 0x18d2bc0] mb P I16..4: 0.9% 13.0% 6.1% P16..4: 32.6% 22.7% 15.8% 0.0% 0.0% skip: 9.0% [libx264 @ 0x18d2bc0] mb B I16..4: 0.1% 2.1% 1.9% B16..8: 39.2% 14.6% 7.8% direct: 4.2% skip:30.0% L0:43.9% L1:36.4% BI:19.7% [libx264 @ 0x18d2bc0] 8x8 transform intra:59.4% inter:60.1% [libx264 @ 0x18d2bc0] coded y,uvDC,uvAC intra: 92.6% 93.1% 68.5% inter: 35.9% 38.0% 9.1% [libx264 @ 0x18d2bc0] i16 v,h,dc,p: 6% 33% 4% 56% [libx264 @ 0x18d2bc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 26% 13% 6% 8% 7% 10% 7% 14% [libx264 @ 0x18d2bc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 28% 10% 7% 9% 7% 11% 6% 11% [libx264 @ 0x18d2bc0] i8c dc,h,v,p: 47% 31% 12% 9% [libx264 @ 0x18d2bc0] Weighted P-Frames: Y:6.1% UV:2.1% [libx264 @ 0x18d2bc0] ref P L0: 62.6% 14.8% 16.1% 6.2% 0.2% [libx264 @ 0x18d2bc0] ref B L0: 80.8% 16.5% 2.7% [libx264 @ 0x18d2bc0] kb/s:12277.17 [aac @ 0x18d0f40] Qavg: 118.000
Stewart
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
Paul B Mahol wrote:
On 2/9/20, S Andreason <sandreas41@gmail.com> wrote:
The issue is with -i gif and overlay=x=(calculations):y=(calculations):between 1. between calculates n differently than the 30fps video, No, it is completely correct, you give it different timebase and fps video inputs. overlay video is 33.33 while main video is 30.00. If you want consistent results make videos same timebase and fps.
The rest all depends on 1. point.
I did not expect the value for n used in between(), to be different than n in x=() and y=(), in the same overlay command.
Use fps filter somehow...
Sure enough, once I add the filter, correctly this time, "fps=fps=30" inserted between [1] and split, the problem is solved. That's the hint I was missing last week, or something... Also problems 3 and 4 have disappeared, so I'll show myself out the door. Stewart Everything takes longer when you get old.
participants (3)
-
Michael Koch -
Paul B Mahol -
S Andreason