[FFmpeg-trac] #3079(FFmpeg:open): libavfilter caches and drops frames with multiple desynched input streams

[FFmpeg]

#3079: libavfilter caches and drops frames with multiple desynched input streams
------------------------------------+----------------------------------
             Reporter:  richardpl   |                    Owner:
                 Type:  defect      |                   Status:  open
             Priority:  normal      |                Component:  FFmpeg
              Version:  git-master  |               Resolution:
             Keywords:  bounty      |               Blocked By:
             Blocking:              |  Reproduced by developer:  1
Analyzed by developer:  1           |
------------------------------------+----------------------------------

Comment (by michael):

 If i understand the issue correctly then a possible solution might be
 * add a source identifier to AVFrame, so that at the filtergraphs output
 as well as in every fifo filter we know precissely from which input how
 many frames are stored. These source identifiers would have to be lists as
 filters like overlay, produce frames that combine 2 sources.
 * Give fifos some means to export these statistics so that we can easily
 check how many frames from input X are in memory for a filtergraph.
 * Use these statistics to guide the inputs, like for example stop a thread
 when many more of its frames are held in memory than another thread.

 With this inputs can run in multiple threads and at different speeds as
 long as it doesnt cause frames to accumulate in buffers, when such
 accumulation occurs due to interconnects in a complex filtergraph than the
 affected input threads would get paused

 The alternative would be to disallow inputs to run at different speeds and
 pause threads that get too far ahead. But this could cause problems if
 timestamps get changed inside the filter graph or future filters send seek
 commands to the source somehow ...

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/3079#comment:18>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker