[FFmpeg-user] Asynchronous overlays

[Alexandre Ferrieux]

On 18/05/2011 22:29, Baptiste Coudurier wrote:
> Hi,
>
> On 05/18/2011 09:24 AM, Alexandre Ferrieux wrote:
>> Hello,
>>
>> I'd like to put side-by-side two video sources (with the overlay
>> filter), with a completely asynchronous/decoupled scheme. Indeed my two
>> video sources have varied and unstable frame rates, and in a naive setup
>> of the overlay filter, ffmpeg insists on fetching a frame from each one
>> for every output frame.
>>
>> The idea is do decode both streams independently (threads) into the same
>> overlay buffer, and have a third thread sample this at a regular rate
>> (the wanted output frame rate), and feed that into the output chain
>> (encoder + container). So, if at any given time one of the sources lags,
>> the same frozen frame from it will be reused in several output frames
>> (ie overlay buffer not updated in that area), but the overall output
>> will not be stalled (as it is today).
>>
>> Q1: is this doable with command-line flags to the ffmpeg executable ?
>
> No.

OK, that's clear at least :)

>> Q2: if not, I'd appreciate a sketch of where to look in the sources to
>> do this in C.
>
> Doing this is way more simple I think, it's just a matter of fetching a
> new overlay frame at the right time depending on pts of the main video,
> I don't think you need any threads here.

Oh yes, everything can always be done single-threadedly with a big select(), but I don't know the overall architecture 
of ffmpeg regarding this. Links appreciated.

In the limit case where there's a single source that is stalled, if we want the output to continue emitting frames at 
OUTPUT-FRAME-PERIOD, we need:

       - either a nonblocking "fetch a new input frame"
       - or a select() monitoring the input with a timeout of OUTPUT-FRAME-PERIOD

Which one is supported by (or in the spirit of) ffmpeg's overall design ?

-Alex