[FFmpeg-devel] [PATCH v2] lavfi/vsrc_ddagrab: add an option to avoid duplicating frames

[Timo Rothenpieler]

On 29.01.2024 19:31, Anton Khirnov wrote:
> Tested-by: Jiří Eliášek, Misha Aizatulin
> ---
> Now requested an infinite timeout from
> IDXGIOutputDuplication_AcquireNextFrame() when a frame is required.
> ---
>   doc/filters.texi           | 15 +++++++++++----
>   libavfilter/vsrc_ddagrab.c |  9 +++++++--
>   2 files changed, 18 insertions(+), 6 deletions(-)
> 
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 1d70f4d934..b9b539acee 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -28713,10 +28713,10 @@ it'll always be captured.
>   @item framerate
>   Maximum framerate at which the desktop will be captured - the interval between
>   successive frames will not be smaller than the inverse of the framerate. When
> -the desktop is not being updated often enough, the filter will duplicate
> -a previous frame. Note that there is no background buffering going on, so when
> -the filter is not polled often enough then the actual inter-frame interval may
> -be significantly larger.
> + at var{dup_frames} is true (the default) and the desktop is not being updated
> +often enough, the filter will duplicate a previous frame. Note that there is no
> +background buffering going on, so when the filter is not polled often enough
> +then the actual inter-frame interval may be significantly larger.
>   
>   Defaults to 30 FPS.
>   
> @@ -28749,6 +28749,13 @@ Passes all supported output formats to DDA and returns what DDA decides to use.
>   Filter initialization will fail if 10 bit format is requested but unavailable.
>   @end table
>   
> + at item dup_frames
> +When this option is set to true (the default), the filter will duplicate frames
> +when the desktop has not been updated in order to maintain approximately
> +constant target framerate. When this option is set to false, the filter will
> +wait for the desktop to be updated (inter-frame intervals may vary significantly
> +in this case).
> +
>   @end table
>   
>   @subsection Examples
> diff --git a/libavfilter/vsrc_ddagrab.c b/libavfilter/vsrc_ddagrab.c
> index 9c59faf53e..7109f5752b 100644
> --- a/libavfilter/vsrc_ddagrab.c
> +++ b/libavfilter/vsrc_ddagrab.c
> @@ -101,6 +101,7 @@ typedef struct DdagrabContext {
>       int        out_fmt;
>       int        allow_fallback;
>       int        force_fmt;
> +    int        dup_frames;
>   } DdagrabContext;
>   
>   #define OFFSET(x) offsetof(DdagrabContext, x)
> @@ -124,6 +125,8 @@ static const AVOption ddagrab_options[] = {
>                                                      OFFSET(allow_fallback), AV_OPT_TYPE_BOOL,   { .i64 = 0    },       0,       1, FLAGS },
>       { "force_fmt",  "exclude BGRA from format list (experimental, discouraged by Microsoft)",
>                                                      OFFSET(force_fmt),  AV_OPT_TYPE_BOOL,       { .i64 = 0    },       0,       1, FLAGS },
> +    { "dup_frames",     "duplicate frames to maintain framerate",
> +                                                   OFFSET(dup_frames), AV_OPT_TYPE_BOOL,       { .i64 = 1    },       0,       1, FLAGS },

nit: align description properly

>       { NULL }
>   };
>   
> @@ -688,7 +691,7 @@ static int next_frame_internal(AVFilterContext *avctx, ID3D11Texture2D **desktop
>   
>       hr = IDXGIOutputDuplication_AcquireNextFrame(
>           dda->dxgi_outdupl,
> -        dda->time_timeout,
> +        need_frame ? INFINITE : dda->time_timeout,

My main worry with this is that the filter could potentially be sitting 
there for a really, really long time.
And also... see below.

>           &frame_info,
>           &desktop_resource);
>       if (hr == DXGI_ERROR_WAIT_TIMEOUT) {
> @@ -1067,7 +1070,9 @@ static int ddagrab_request_frame(AVFilterLink *outlink)
>       now -= dda->first_pts;
>   
>       if (!dda->probed_texture) {
> -        ret = next_frame_internal(avctx, &cur_texture, 0);
> +        do {
> +            ret = next_frame_internal(avctx, &cur_texture, !dda->dup_frames);
> +        } while (ret == AVERROR(EAGAIN) && !dda->dup_frames);

This is unfortunately not quiet correct, as it misunderstands the 
need_frame parameter.
The need_frame parameter is meant for the initial probing period, where 
an actual _frame_ is needed, outside of the probing, mouse-location-only 
updates are perfectly valid though.

So with this implementation, if the desktop remains static, but the 
mouse is moved, this loop will keep asking for a frame, and not record 
the mouse movement.

My idea for a fix would be to just pass 0 to need_frame as it was, but 
keep the outer loop like this.
And either make the INFINITE timeout depend on the dup_frames option 
directly, or just don't pass INFINITE, and let it loop.

>       } else {
>           cur_texture = dda->probed_texture;
>           dda->probed_texture = NULL;