[FFmpeg-devel] [PATCH] frame durations for framesync

[Nicolas George]

Paul B Mahol (12023-01-27):
> From b4f835c4ef6e0e0bbe6adef8235381e56f3f91df Mon Sep 17 00:00:00 2001
> From: Paul B Mahol <onemda at gmail.com>
> Date: Fri, 27 Jan 2023 23:34:02 +0100
> Subject: [PATCH 1/4] avfilter/framesync: calculate frame duration too
> 
> Signed-off-by: Paul B Mahol <onemda at gmail.com>
> ---
>  libavfilter/framesync.c | 20 ++++++++++++++++----
>  libavfilter/framesync.h | 15 +++++++++++++++
>  2 files changed, 31 insertions(+), 4 deletions(-)

It does not work. For example, with the following filter graph:

testsrc2=r=1/3[a];testsrc2=r=1/5[b];[a][b]vstack

and adding a line of debug at the beginning of process_frame() in
vf_stack.c, you get:

got event: pts = 0, duration = 3
got event: pts = 3, duration = 3 ← should be 2
got event: pts = 5, duration = 5 ← should be 1
got event: pts = 6, duration = 3 …
got event: pts = 9, duration = 3
got event: pts = 10, duration = 5
got event: pts = 12, duration = 3
got event: pts = 15, duration = 3
got event: pts = 18, duration = 3
got event: pts = 20, duration = 5

I was wrong saying it cannot work, you can compute the duration of the
new frame without increasing the latency, but you have to duplicate the
whole frame synchronization logic.

I was against adding the duration field to libavfilter, and this is a
very good illustration of the reason. The duration field is redundant
with other information, it takes a lot of code to maintain it, and we
cannot trust it anyway.

For libavfilter, my advice would be to disregard it entirely. Let us
just clear it in buffersrc and be done with it.

Regards,

-- 
  Nicolas George