#8589(undetermined:new): Concat demuxer is not closing file handles for old playlists
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Type: defect Status: new | Priority: normal Component: | Version: 4.2 undetermined | Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- == Summary of the bug: == on Ubuntu 18.04 (i guess on all linux distributions), when you reference a new playlist in a concat playlist file and this new playlist is getting played the file handle of the first playlist is not getting closed, even when the second playlist has the same name. == How to reproduce: == generate a test-video like this: {{{ ffmpeg -f lavfi -i testsrc=s=720x576:r=12:d=4 -pix_fmt yuv422p -vf "drawbox=w=50:h=w:t=w:c=${c:?}" test.mkv }}} generate a playlist file called "test.txt" with this content: {{{ ffconcat version 1.0 file 'test.mkv' file 'test.txt' }}} invoke ffmpeg like this (streaming the file to a local udp port just to get it going forever): {{{ ffmpeg -re -f concat -i test.txt -c copy -f rtp rtp://localhost:2000 }}} Get the process number for ffmpeg (like ps xa|grep ffmpeg) Check how many file handles this process takes: {{{ lsof -p <processnumber> }}} You will see that over time the "test.txt" playlist file handle will get more and more, so the concat demuxer is not closing the file handle after reading the playlist. Same behaviour if you switch between different playlists or use pipes, even the bash example on https://trac.ffmpeg.org/wiki/Concatenate has this problem. Over time this leads to cpu increase of the ffmpeg process until the machine runs out of file handles. I tried it with different ffmpeg version from 4.1 to 4.2 (including on Ubuntu the version from the ppa:jonathonf/ffmpeg-4 repository) every time the same problem. -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Owner: Type: defect | Status: new Priority: normal | Component: | undetermined Version: 4.2 | Resolution: Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by cus): What do you expect if you reference the playlist file in the playlist? -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:1> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Owner: Type: defect | Status: new Priority: normal | Component: | undetermined Version: 4.2 | Resolution: Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by karli2000): Well, the example references the same video over and over again and the file handle for the video is always getting closed every time when the video is finished, i would excpect the same for the playlist-file. And, when you make 10 playlist-files and chain them together so that playlist 1 is referencing playlist 2 and so on the file handle for playlist 1, even hours after it was used (all videos processed from this play list) is not getting closed, but the file handles for all videos that ffmpeg processed from that playlist are closed. -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:2> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Owner: Type: defect | Status: new Priority: normal | Component: | undetermined Version: 4.2 | Resolution: Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by cus): I guess you are referring to the "Automatically appending to the list file" example on the wiki, right? The problem is that you are crearing an infinite recursion of playlists. That will eventually run out of resources, file descriptors first in your case. FFmpeg only closes the playlist source file descriptor when that playlist finishes. None of the playlist files finishes, because the last entry is another playlist which never finishes, and so on. If you need simple loop features, use -stream_loop, and not this endless playlist recursion, which is a *if it works it is not stupid* kind of hack and has its limitations. -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:3> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Owner: Type: defect | Status: new Priority: normal | Component: | undetermined Version: 4.2 | Resolution: Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by karli2000): I understand what you are saying, but i still think this is a bug or at least it should be mentioned somewhere. Using playlists is the only way i found to dynamically feed new videos (by chaining playlists) to an existing, permanent running output-stream, -stream_loop is only for looping the same video. -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:4> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Owner: Type: defect | Status: new Priority: normal | Component: | undetermined Version: 4.2 | Resolution: Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by cus): This may work for you as well depending on use case: dummy.txt: {{{ ffconcat version 1.0 file dummy.mkv file dummy.mkv }}} {{{ ffmpeg -re -stream_loop -1 -i dummy.txt -f sdl2 none }}} You can seamlessly change the input by atomically replacing dummy.mkv. The wiki should probably be updated that the chaining method has limitations. -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:5> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Owner: Type: defect | Status: new Priority: normal | Component: | undetermined Version: 4.2 | Resolution: Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Comment (by karli2000): Did not thought of this, nice idea, will try it! Yes please, the wiki should be updated for this. -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:6> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------- Reporter: karli2000 | Owner: Type: defect | Status: closed Priority: normal | Component: | undetermined Version: 4.2 | Resolution: wontfix Keywords: concat | Blocked By: playlist cpu | Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------- Changes (by cus): * status: new => closed * resolution: => wontfix Comment: I have updated the wiki, so closing this as wontfix. -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:7> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#8589: Concat demuxer is not closing file handles for old playlists -------------------------------------+------------------------------------ Reporter: karli2000 | Owner: Type: defect | Status: closed Priority: normal | Component: avformat Version: unspecified | Resolution: wontfix Keywords: concat | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------ Changes (by cehoyos): * keywords: concat playlist cpu => concat * version: 4.2 => unspecified * component: undetermined => avformat -- Ticket URL: <https://trac.ffmpeg.org/ticket/8589#comment:8> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
participants (1)
-
FFmpeg