[PATCH, v2] lavf/vf_deinterlace_vaapi: flush queued frame for field in DeinterlacingBob
For DeinterlacingBob mode with rate=field, the frame number of output should equal 2x input total since only intra deinterlace is used. Currently for "backward_ref = 0, rate = field", extra_delay is introduced. Due to the async without flush, frame number of output is [expected_number - 2]. Specifically, if the input only has 1 frame, the output will be empty. Add deint_vaapi_request_frame for deinterlace_vaapi, send NULL frame to flush the queued frame. For 1 frame input in Bob mode with rate=field, before patch: 0 frame; after patch: 2 frames; ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.h264 -an -vf deinterlace_vaapi=mode=bob:rate=field -f null - Signed-off-by: Linjie Fu <linjie.fu@intel.com> --- libavfilter/vf_deinterlace_vaapi.c | 44 ++++++++++++++++++++++++++++++++------ 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/libavfilter/vf_deinterlace_vaapi.c b/libavfilter/vf_deinterlace_vaapi.c index 72d0349..fe77118 100644 --- a/libavfilter/vf_deinterlace_vaapi.c +++ b/libavfilter/vf_deinterlace_vaapi.c @@ -48,6 +48,9 @@ typedef struct DeintVAAPIContext { int queue_count; AVFrame *frame_queue[MAX_REFERENCES]; int extra_delay_for_timestamps; + + int eof; + int prev_pts; } DeintVAAPIContext; static const char *deint_vaapi_mode_name(int mode) @@ -190,9 +193,11 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) void *filter_params_addr = NULL; int err, i, field, current_frame_index; - av_log(avctx, AV_LOG_DEBUG, "Filter input: %s, %ux%u (%"PRId64").\n", - av_get_pix_fmt_name(input_frame->format), - input_frame->width, input_frame->height, input_frame->pts); + // NULL frame is used to flush the queue in field mode + if (input_frame) + av_log(avctx, AV_LOG_DEBUG, "Filter input: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(input_frame->format), + input_frame->width, input_frame->height, input_frame->pts); if (ctx->queue_count < ctx->queue_depth) { ctx->frame_queue[ctx->queue_count++] = input_frame; @@ -210,6 +215,9 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) current_frame_index = ctx->pipeline_caps.num_forward_references; input_frame = ctx->frame_queue[current_frame_index]; + if (!input_frame) + return 0; + input_surface = (VASurfaceID)(uintptr_t)input_frame->data[3]; for (i = 0; i < ctx->pipeline_caps.num_forward_references; i++) forward_references[i] = (VASurfaceID)(uintptr_t) @@ -291,6 +299,8 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) if (ctx->field_rate == 2) { if (field == 0) output_frame->pts = 2 * input_frame->pts; + else if (ctx->eof) + output_frame->pts = 3 * input_frame->pts - ctx->prev_pts; else output_frame->pts = input_frame->pts + ctx->frame_queue[current_frame_index + 1]->pts; @@ -306,6 +316,8 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) break; } + ctx->prev_pts = input_frame->pts; + return err; fail: @@ -315,6 +327,25 @@ fail: return err; } +static int deint_vaapi_request_frame(AVFilterLink *link) +{ + AVFilterContext *avctx = link->src; + DeintVAAPIContext *ctx = avctx->priv; + int ret; + + if (ctx->eof) + return AVERROR_EOF; + + ret = ff_request_frame(link->src->inputs[0]); + if (ret == AVERROR_EOF && ctx->extra_delay_for_timestamps) { + ctx->eof = 1; + deint_vaapi_filter_frame(link->src->inputs[0], NULL); + } else if (ret < 0) + return ret; + + return 0; +} + static av_cold int deint_vaapi_init(AVFilterContext *avctx) { VAAPIVPPContext *vpp_ctx = avctx->priv; @@ -376,9 +407,10 @@ static const AVFilterPad deint_vaapi_inputs[] = { static const AVFilterPad deint_vaapi_outputs[] = { { - .name = "default", - .type = AVMEDIA_TYPE_VIDEO, - .config_props = &deint_vaapi_config_output, + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .request_frame = &deint_vaapi_request_frame, + .config_props = &deint_vaapi_config_output, }, { NULL } }; -- 2.7.4
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Linjie Fu Sent: Wednesday, September 18, 2019 16:20 To: ffmpeg-devel@ffmpeg.org Cc: Fu, Linjie <linjie.fu@intel.com> Subject: [FFmpeg-devel] [PATCH, v2] lavf/vf_deinterlace_vaapi: flush queued frame for field in DeinterlacingBob
For DeinterlacingBob mode with rate=field, the frame number of output should equal 2x input total since only intra deinterlace is used.
Currently for "backward_ref = 0, rate = field", extra_delay is introduced. Due to the async without flush, frame number of output is [expected_number - 2].
Specifically, if the input only has 1 frame, the output will be empty.
Add deint_vaapi_request_frame for deinterlace_vaapi, send NULL frame to flush the queued frame.
For 1 frame input in Bob mode with rate=field, before patch: 0 frame; after patch: 2 frames;
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.h264 -an -vf deinterlace_vaapi=mode=bob:rate=field -f null -
Sorry for picking up the old thread. Hi Mark, Is your issue mentioned in http://ffmpeg.org/pipermail/ffmpeg-devel/2019-September/250080.html fixed by this patch version ? I'd like to apply this patch if it may work for you. Thanks Haihao
Signed-off-by: Linjie Fu <linjie.fu@intel.com> --- libavfilter/vf_deinterlace_vaapi.c | 44 ++++++++++++++++++++++++++++++++------ 1 file changed, 38 insertions(+), 6 deletions(-)
diff --git a/libavfilter/vf_deinterlace_vaapi.c b/libavfilter/vf_deinterlace_vaapi.c index 72d0349..fe77118 100644 --- a/libavfilter/vf_deinterlace_vaapi.c +++ b/libavfilter/vf_deinterlace_vaapi.c @@ -48,6 +48,9 @@ typedef struct DeintVAAPIContext { int queue_count; AVFrame *frame_queue[MAX_REFERENCES]; int extra_delay_for_timestamps; + + int eof; + int prev_pts; } DeintVAAPIContext;
static const char *deint_vaapi_mode_name(int mode) @@ -190,9 +193,11 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) void *filter_params_addr = NULL; int err, i, field, current_frame_index;
- av_log(avctx, AV_LOG_DEBUG, "Filter input: %s, %ux%u (%"PRId64").\n", - av_get_pix_fmt_name(input_frame->format), - input_frame->width, input_frame->height, input_frame->pts); + // NULL frame is used to flush the queue in field mode + if (input_frame) + av_log(avctx, AV_LOG_DEBUG, "Filter input: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(input_frame->format), + input_frame->width, input_frame->height, input_frame->pts);
if (ctx->queue_count < ctx->queue_depth) { ctx->frame_queue[ctx->queue_count++] = input_frame; @@ -210,6 +215,9 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) current_frame_index = ctx->pipeline_caps.num_forward_references;
input_frame = ctx->frame_queue[current_frame_index]; + if (!input_frame) + return 0; + input_surface = (VASurfaceID)(uintptr_t)input_frame->data[3]; for (i = 0; i < ctx->pipeline_caps.num_forward_references; i++) forward_references[i] = (VASurfaceID)(uintptr_t) @@ -291,6 +299,8 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) if (ctx->field_rate == 2) { if (field == 0) output_frame->pts = 2 * input_frame->pts; + else if (ctx->eof) + output_frame->pts = 3 * input_frame->pts - ctx->prev_pts; else output_frame->pts = input_frame->pts + ctx->frame_queue[current_frame_index + 1]->pts; @@ -306,6 +316,8 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) break; }
+ ctx->prev_pts = input_frame->pts; + return err;
fail: @@ -315,6 +327,25 @@ fail: return err; }
+static int deint_vaapi_request_frame(AVFilterLink *link) +{ + AVFilterContext *avctx = link->src; + DeintVAAPIContext *ctx = avctx->priv; + int ret; + + if (ctx->eof) + return AVERROR_EOF; + + ret = ff_request_frame(link->src->inputs[0]); + if (ret == AVERROR_EOF && ctx->extra_delay_for_timestamps) { + ctx->eof = 1; + deint_vaapi_filter_frame(link->src->inputs[0], NULL); + } else if (ret < 0) + return ret; + + return 0; +} + static av_cold int deint_vaapi_init(AVFilterContext *avctx) { VAAPIVPPContext *vpp_ctx = avctx->priv; @@ -376,9 +407,10 @@ static const AVFilterPad deint_vaapi_inputs[] = {
static const AVFilterPad deint_vaapi_outputs[] = { { - .name = "default", - .type = AVMEDIA_TYPE_VIDEO, - .config_props = &deint_vaapi_config_output, + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .request_frame = &deint_vaapi_request_frame, + .config_props = &deint_vaapi_config_output, }, { NULL } }; -- 2.7.4
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 29/12/2021 03:45, Xiang, Haihao wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Linjie Fu Sent: Wednesday, September 18, 2019 16:20 To: ffmpeg-devel@ffmpeg.org Cc: Fu, Linjie <linjie.fu@intel.com> Subject: [FFmpeg-devel] [PATCH, v2] lavf/vf_deinterlace_vaapi: flush queued frame for field in DeinterlacingBob
For DeinterlacingBob mode with rate=field, the frame number of output should equal 2x input total since only intra deinterlace is used.
Currently for "backward_ref = 0, rate = field", extra_delay is introduced. Due to the async without flush, frame number of output is [expected_number - 2].
Specifically, if the input only has 1 frame, the output will be empty.
Add deint_vaapi_request_frame for deinterlace_vaapi, send NULL frame to flush the queued frame.
For 1 frame input in Bob mode with rate=field, before patch: 0 frame; after patch: 2 frames;
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.h264 -an -vf deinterlace_vaapi=mode=bob:rate=field -f null -
Sorry for picking up the old thread.
Hi Mark,
Is your issue mentioned in http://ffmpeg.org/pipermail/ffmpeg-devel/2019-September/250080.html fixed by this patch version ? I'd like to apply this patch if it may work for you.
Yep, it now gives the expected number of frames and I do not have any segfaults. I tested drivers with { forward = 0, backward = 0 } (iHD), { forward = 1, backward = 0 } (i965) and { forward = 2, backward = 1 } (gallium). Thanks, - Mark
On Sun, 2022-01-09 at 18:32 +0000, Mark Thompson wrote:
On 29/12/2021 03:45, Xiang, Haihao wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Linjie Fu Sent: Wednesday, September 18, 2019 16:20 To: ffmpeg-devel@ffmpeg.org Cc: Fu, Linjie <linjie.fu@intel.com> Subject: [FFmpeg-devel] [PATCH, v2] lavf/vf_deinterlace_vaapi: flush queued frame for field in DeinterlacingBob
For DeinterlacingBob mode with rate=field, the frame number of output should equal 2x input total since only intra deinterlace is used.
Currently for "backward_ref = 0, rate = field", extra_delay is introduced. Due to the async without flush, frame number of output is [expected_number - 2].
Specifically, if the input only has 1 frame, the output will be empty.
Add deint_vaapi_request_frame for deinterlace_vaapi, send NULL frame to flush the queued frame.
For 1 frame input in Bob mode with rate=field, before patch: 0 frame; after patch: 2 frames;
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.h264 -an -vf deinterlace_vaapi=mode=bob:rate=field -f null -
Sorry for picking up the old thread.
Hi Mark,
Is your issue mentioned in http://ffmpeg.org/pipermail/ffmpeg-devel/2019-September/250080.html fixed by this patch version ? I'd like to apply this patch if it may work for you.
Yep, it now gives the expected number of frames and I do not have any segfaults.
I tested drivers with { forward = 0, backward = 0 } (iHD), { forward = 1, backward = 0 } (i965) and { forward = 2, backward = 1 } (gallium).
Thanks,
Thanks for confirming it works, and I will apply this patch version if no objection. -Haihao
On Mon, 2022-01-10 at 06:58 +0000, Xiang, Haihao wrote:
On Sun, 2022-01-09 at 18:32 +0000, Mark Thompson wrote:
On 29/12/2021 03:45, Xiang, Haihao wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Linjie Fu Sent: Wednesday, September 18, 2019 16:20 To: ffmpeg-devel@ffmpeg.org Cc: Fu, Linjie <linjie.fu@intel.com> Subject: [FFmpeg-devel] [PATCH, v2] lavf/vf_deinterlace_vaapi: flush queued frame for field in DeinterlacingBob
For DeinterlacingBob mode with rate=field, the frame number of output should equal 2x input total since only intra deinterlace is used.
Currently for "backward_ref = 0, rate = field", extra_delay is introduced. Due to the async without flush, frame number of output is [expected_number - 2].
Specifically, if the input only has 1 frame, the output will be empty.
Add deint_vaapi_request_frame for deinterlace_vaapi, send NULL frame to flush the queued frame.
For 1 frame input in Bob mode with rate=field, before patch: 0 frame; after patch: 2 frames;
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.h264 -an -vf deinterlace_vaapi=mode=bob:rate=field -f null -
Sorry for picking up the old thread.
Hi Mark,
Is your issue mentioned in http://ffmpeg.org/pipermail/ffmpeg-devel/2019-September/250080.html fixed by this patch version ? I'd like to apply this patch if it may work for you.
Yep, it now gives the expected number of frames and I do not have any segfaults.
I tested drivers with { forward = 0, backward = 0 } (iHD), { forward = 1, backward = 0 } (i965) and { forward = 2, backward = 1 } (gallium).
Thanks,
Thanks for confirming it works, and I will apply this patch version if no objection.
Applied, thx! -Haihao
participants (3)
-
Linjie Fu -
Mark Thompson -
Xiang, Haihao