[FFmpeg-trac] #10932(undetermined:new): Regression with filter graph involving -shortest flag

[FFmpeg]

#10932: Regression with filter graph involving -shortest flag
-------------------------------------+-------------------------------------
             Reporter:  Wes Castro   |                     Type:  defect
               Status:  new          |                 Priority:  important
            Component:               |                  Version:  git-
  undetermined                       |  master
             Keywords:  shortest     |               Blocked By:
  aloop hang regression              |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:

 A command line that worked with FFmpeg 5.1 produces a regressed output in
 FFmpeg 6.0, 6.1, and git-master. The expected output is a ~15 second MP4
 with a video and audio stream. With git-master all video frames are
 dropped and only a 980ms audio stream is produced.

 Apologies for the large filter graph... when attempting to reduce the
 chain I was unable to reproduce the issue in the same way.

 How to reproduce:

 === Generate test inputs:
 FFmpeg for all commands built with only `./configure --enable-zlib && make
 -j$(nproc)`
 {{{
 ffmpeg -y -f lavfi -i testsrc=duration=15:size=1080x1920:rate=30 -f lavfi
 -i sine=frequency=1000:sample_rate=48000:duration=14.9
 15secondvideoaudio.mp4
 ffmpeg -y -f lavfi -i sine=frequency=1000:sample_rate=48000:duration=300
 5minaudio.mp4
 ffmpeg -y -f lavfi -i testsrc=size=1280x720 -frames:v 1 image.png
 }}}

 === Run problematic filter graph:
 {{{
 ffmpeg -v 9 -loglevel 99 -hide_banner -y -i 15secondvideoaudio.mp4 -ss 21
 -to 35 -i 5minaudio.mp4 -i 15secondvideoaudio.mp4 -framerate 30 -i
 image.png -framerate 30 -i image.png -filter_complex "[2:v]
 scale=w=1080:h=-2 [_fg_scaled] ; nullsrc=size=1080x1920:r=2:d=1,
 geq=red_expr='88+(Y/H)*(72-88)':green_expr='136+(Y/H)*(72-136)':blue_expr='184+(Y/H)*(72-184)',
 loop=-1:size=2, fps=fps=30 [_bg] ; [_bg][_fg_scaled] overlay=(main_w-
 overlay_w)/2:(main_h-overlay_h)/2:shortest=1
 [uploaded_video_2_0_uploaded_video] ;
 [3:v][uploaded_video_2_0_uploaded_video]
 scale2ref=w=in_w*1.000000:h=in_h*1.000000:sws_flags=lanczos
 [_overlay][_video_in] ; [_video_in][_overlay]
 overlay=x=main_w*0.000000:y=main_h*0.000000:shortest=0:repeatlast=1
 [video1] ; [4:v][video1]
 scale2ref=w=in_w*0.430556:h=in_h*0.075000:sws_flags=lanczos
 [_overlay][_video_in] ; [_video_in][_overlay]
 overlay=x=main_w*0.234429:y=main_h*-0.132120:shortest=0:repeatlast=1
 [_video_out] ; [_video_out] setpts=PTS-STARTPTS [video]" -filter_complex
 "[1:a] volume=volume=1.000, afade=type=in:start_time=0:duration=0.4,
 afade=type=out:start_time=14.367:duration=0.6,
 aloop=loop=-1:size=2.14748e+09 [music] ; [0:a] volume=volume=0.000
 [original_audio] ; [original_audio][music] amix=inputs=2:duration=longest,
 asetpts=PTS-STARTPTS [music_mix]" -map [music_mix] -map [video]
 -map_metadata -1 -shortest -f mp4 -y output.mp4
 }}}

 === Results
 - FFmpeg 5.1
   - Produces 15.04 second video+audio
 - FFmpeg 6.0
  - Hangs indefinitely when encoding frame 446 (when I cancel it with
 Ctrl-C the last `frame=` line output is for 448)
 - FFmpeg 6.1
  - Fails with `Error while filtering: No space left`
  - Workaround: Change loop=-1 to loop=99
   - Not ideal since loop is no longer infinite
   - Higher loop value (999) will cause conversion to fail with `Error
 while filtering: No space left` again
 - FFmpeg master branch 3/26/2024
 (9df11820650b1d93f6a49519aaa51afb5cc17086)
   - Warning is printed: `[vost#0:1/mpeg4 @ 0x2e90500] No filtered frames
 for output stream, trying to initialize anyway.`
   - Produces no video frames and 980ms of audio.

 === Notes
 Changing the audio stream duration of `15secondvideoaudio.mp4` to 14.8 or
 15.0 causes the hang to not occur in FFmpeg 6.0 or 6.1, but the master
 branch issue still occurs.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10932>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker