February 2021 - ffmpeg-devel - lists.ffmpeg.org

[PATCH v2] avformat/webvttdec, enc: correctly process files containing STYLE, REGION blocks
by Dave Evans 24 Nov '21

24 Nov '21

This patch fixes the total failure to parse cues when style and region definition blocks are contained in the input file, and ensures those blocks are written to the output when copying. The sample attached needs to be added to samples at the path shown in the patch in order to validate that the original issue is fixed. Same as v1 except the test has been changed as requested in the original review. Cheers, Dave

1 2

[PATCH v1] avfilter: add overlay vaapi filter
by Fei Wang 22 Nov '21

22 Nov '21

From: Xinpeng Sun <xinpeng.sun(a)intel.com> Overlay one video on the top of another. It takes two inputs and has one output. The first input is the "main" video on which the second input is overlaid. This filter requires same memory layout for all the inputs. An example command to use this filter to overlay an image LOGO at the top-left corner of the INPUT video and both inputs are h264 file: ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 \ -hwaccel_output_format vaapi -i INPUT.h264 -hwaccel vaapi \ -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i \ LOGO.h264 -filter_complex "[0:v][1:v]overlay_vaapi=w=50:h=50:alpha=0.5" \ -c:v h264_vaapi -y OUTPUT.h264 Signed-off-by: Xinpeng Sun <xinpeng.sun(a)intel.com> Signed-off-by: Zachary Zhou <zachary.zhou(a)intel.com> Signed-off-by: Fei Wang <fei.w.wang(a)intel.com> --- Changelog | 1 + configure | 3 + doc/filters.texi | 51 ++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 2 +- libavfilter/vf_overlay_vaapi.c | 426 +++++++++++++++++++++++++++++++++ 7 files changed, 484 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vf_overlay_vaapi.c diff --git a/Changelog b/Changelog index a60e7d2eb8..f745f6f52a 100644 --- a/Changelog +++ b/Changelog @@ -4,6 +4,7 @@ releases are sorted from youngest to oldest. version <next>: - AudioToolbox output device - MacCaption demuxer +- overlay_vaapi filter version 4.3: diff --git a/configure b/configure index 7495f35faa..c930b261a2 100755 --- a/configure +++ b/configure @@ -3560,6 +3560,7 @@ openclsrc_filter_deps="opencl" overlay_opencl_filter_deps="opencl" overlay_qsv_filter_deps="libmfx" overlay_qsv_filter_select="qsvvpp" +overlay_vaapi_filter_deps="vaapi" overlay_vulkan_filter_deps="vulkan libglslang" owdenoise_filter_deps="gpl" pad_opencl_filter_deps="opencl" @@ -3620,6 +3621,7 @@ tonemap_vaapi_filter_deps="vaapi VAProcFilterParameterBufferHDRToneMapping" tonemap_opencl_filter_deps="opencl const_nan" transpose_opencl_filter_deps="opencl" transpose_vaapi_filter_deps="vaapi VAProcPipelineCaps_rotation_flags" +overlay_vaapi_filter_deps="vaapi VAProcPipelineCaps_blend_flags" unsharp_opencl_filter_deps="opencl" uspp_filter_deps="gpl avcodec" vaguedenoiser_filter_deps="gpl" @@ -6643,6 +6645,7 @@ if enabled vaapi; then check_struct "va/va.h" "VADecPictureParameterBufferVP9" bit_depth check_type "va/va.h va/va_vpp.h" "VAProcFilterParameterBufferHDRToneMapping" check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" rotation_flags + check_struct "va/va.h va/va_vpp.h" "VAProcPipelineCaps" blend_flags check_type "va/va.h va/va_enc_hevc.h" "VAEncPictureParameterBufferHEVC" check_type "va/va.h va/va_enc_jpeg.h" "VAEncPictureParameterBufferJPEG" check_type "va/va.h va/va_enc_vp8.h" "VAEncPictureParameterBufferVP8" diff --git a/doc/filters.texi b/doc/filters.texi index 67892e0afb..c53fe37bae 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -22128,6 +22128,57 @@ To enable compilation of these filters you need to configure FFmpeg with To use vaapi filters, you need to setup the vaapi device correctly. For more information, please read @url{https://trac.ffmpeg.org/wiki/Hardware/VAAPI} +@section overlay_vaapi + +Overlay one video on the top of another. + +It takes two inputs and has one output. The first input is the "main" video on which the second input is overlaid. +This filter requires same memory layout for all the inputs. So, format conversion may be needed. + +The filter accepts the following options: + +@table @option + +@item x +Set the x coordinate of the overlaid video on the main video. +Default value is @code{0}. + +@item y +Set the y coordinate of the overlaid video on the main video. +Default value is @code{0}. + +@item w +Set the width of the overlaid video on the main video. +Default value is the width of input overlay video. + +@item h +Set the height of the overlaid video on the main video. +Default value is the height of input overlay video. + +@item alpha +Set blocking detection thresholds. Allowed range is 0.0 to 1.0, it +requires an input video with alpha channel. +Default value is @code{0.0}. + +@end table + +@subsection Examples + +@itemize +@item +Overlay an image LOGO at the top-left corner of the INPUT video. Both inputs for this filter are yuv420p format. +@example +-i INPUT -i LOGO -filter_complex "[0:v]hwupload[a], [1:v]format=yuv420p, hwupload[b], [a][b]overlay_vaapi" OUTPUT +@end example +@item +Overlay an image LOGO at the offset (200, 100) from the top-left corner of the INPUT video. +The inputs have same memory layout for color channels, the overlay has additional alpha plane, like INPUT is yuv420p, and the LOGO is yuva420p. +@example +-i INPUT -i LOGO -filter_complex "[0:v]hwupload[a], [1:v]format=yuva420p, hwupload[b], [a][b]overlay_vaapi=x=200:y=100:w=400:h=300:alpha=1.0, hwdownload, format=nv12" OUTPUT +@end example + +@end itemize + @section tonemap_vaapi Perform HDR(High Dynamic Range) to SDR(Standard Dynamic Range) conversion with tone-mapping. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 5123540653..c991f9ba7c 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -331,6 +331,7 @@ OBJS-$(CONFIG_OVERLAY_CUDA_FILTER) += vf_overlay_cuda.o framesync.o vf OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ opencl/overlay.o framesync.o OBJS-$(CONFIG_OVERLAY_QSV_FILTER) += vf_overlay_qsv.o framesync.o +OBJS-$(CONFIG_OVERLAY_VAAPI_FILTER) += vf_overlay_vaapi.o framesync.o vaapi_vpp.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER) += vf_pad.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 1183e40267..8a149f90a9 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -314,6 +314,7 @@ extern AVFilter ff_vf_oscilloscope; extern AVFilter ff_vf_overlay; extern AVFilter ff_vf_overlay_opencl; extern AVFilter ff_vf_overlay_qsv; +extern AVFilter ff_vf_overlay_vaapi; extern AVFilter ff_vf_overlay_vulkan; extern AVFilter ff_vf_overlay_cuda; extern AVFilter ff_vf_owdenoise; diff --git a/libavfilter/version.h b/libavfilter/version.h index 37015085fa..281a9d8b03 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -31,7 +31,7 @@ #define LIBAVFILTER_VERSION_MAJOR 7 #define LIBAVFILTER_VERSION_MINOR 86 -#define LIBAVFILTER_VERSION_MICRO 100 +#define LIBAVFILTER_VERSION_MICRO 101 #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \ diff --git a/libavfilter/vf_overlay_vaapi.c b/libavfilter/vf_overlay_vaapi.c new file mode 100644 index 0000000000..993db9148e --- /dev/null +++ b/libavfilter/vf_overlay_vaapi.c @@ -0,0 +1,426 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ +#include <string.h> + +#include "libavutil/avassert.h" +#include "libavutil/mem.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" + +#include "avfilter.h" +#include "framesync.h" +#include "formats.h" +#include "internal.h" +#include "vaapi_vpp.h" + +typedef struct OverlayVAAPIContext { + VAAPIVPPContext vpp_ctx; // must be the first field + FFFrameSync fs; + int overlay_ox; + int overlay_oy; + int overlay_ow; + int overlay_oh; + float alpha; +} OverlayVAAPIContext; + +static int overlay_vaapi_query_formats(AVFilterContext *ctx) +{ + int ret; + enum { + MAIN = 0, + OVERLAY = 1, + }; + + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_VAAPI, + AV_PIX_FMT_NONE + }; + + ret = ff_formats_ref(ff_make_format_list(pix_fmts), &ctx->inputs[MAIN]->out_formats); + if (ret < 0) + return ret; + + ret = ff_formats_ref(ff_make_format_list(pix_fmts), &ctx->inputs[OVERLAY]->out_formats); + if (ret < 0) + return ret; + + ret = ff_formats_ref(ff_make_format_list(pix_fmts), &ctx->outputs[0]->in_formats); + if (ret < 0) + return ret; + + return 0; +} + +static int overlay_vaapi_build_filter_params(AVFilterContext *avctx) +{ + VAAPIVPPContext *vpp_ctx = avctx->priv; + VAStatus vas; + int support_flag; + VAProcPipelineCaps pipeline_caps; + + memset(&pipeline_caps, 0, sizeof(pipeline_caps)); + vas = vaQueryVideoProcPipelineCaps(vpp_ctx->hwctx->display, + vpp_ctx->va_context, + NULL, 0, + &pipeline_caps); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to query pipeline " + "caps: %d (%s).\n", vas, vaErrorStr(vas)); + return AVERROR(EIO); + } + + if (!pipeline_caps.blend_flags) { + av_log(avctx, AV_LOG_ERROR, "VAAPI driver doesn't support overlay\n"); + return AVERROR(EINVAL); + } + + support_flag = pipeline_caps.blend_flags & VA_BLEND_GLOBAL_ALPHA; + if (!support_flag) { + av_log(avctx, AV_LOG_ERROR, "VAAPI driver doesn't support global alpha blending\n"); + return AVERROR(EINVAL); + } + + return 0; +} + + +static int overlay_vaapi_render_picture(AVFilterContext *avctx, + VAProcPipelineParameterBuffer *params, + VAProcPipelineParameterBuffer *subpic_params, + AVFrame *output_frame) +{ + VAAPIVPPContext *ctx = avctx->priv; + VASurfaceID output_surface; + VABufferID params_id; + VABufferID subpic_params_id; + VAStatus vas; + int err = 0; + + output_surface = (VASurfaceID)(uintptr_t)output_frame->data[3]; + + vas = vaBeginPicture(ctx->hwctx->display, + ctx->va_context, output_surface); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to attach new picture: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + + vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, + VAProcPipelineParameterBufferType, + sizeof(*params), 1, params, &params_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to create parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail_after_begin; + } + av_log(avctx, AV_LOG_DEBUG, "Pipeline parameter buffer is %#x.\n", + params_id); + + + vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, + VAProcPipelineParameterBufferType, + sizeof(*subpic_params), 1, subpic_params, &subpic_params_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to create parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail_after_begin; + } + av_log(avctx, AV_LOG_DEBUG, "Pipeline subpic parameter buffer is %#x.\n", + subpic_params_id); + + vas = vaRenderPicture(ctx->hwctx->display, ctx->va_context, + &params_id, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to render parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail_after_begin; + } + + vas = vaRenderPicture(ctx->hwctx->display, ctx->va_context, + &subpic_params_id, 1); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to render subpic parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail_after_begin; + } + + vas = vaEndPicture(ctx->hwctx->display, ctx->va_context); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to start picture processing: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail_after_render; + } + + if (CONFIG_VAAPI_1 || ctx->hwctx->driver_quirks & + AV_VAAPI_DRIVER_QUIRK_RENDER_PARAM_BUFFERS) { + vas = vaDestroyBuffer(ctx->hwctx->display, params_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to free parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + // And ignore. + } + } + + return 0; + + // We want to make sure that if vaBeginPicture has been called, we also + // call vaRenderPicture and vaEndPicture. These calls may well fail or + // do something else nasty, but once we're in this failure case there + // isn't much else we can do. +fail_after_begin: + vaRenderPicture(ctx->hwctx->display, ctx->va_context, &params_id, 1); +fail_after_render: + vaEndPicture(ctx->hwctx->display, ctx->va_context); +fail: + return err; +} + +static int overlay_vaapi_blend(FFFrameSync *fs) +{ + AVFilterContext *avctx = fs->parent; + AVFilterLink *outlink = avctx->outputs[0]; + OverlayVAAPIContext *ctx = avctx->priv; + VAAPIVPPContext *vpp_ctx = avctx->priv; + AVFrame *input_main, *input_overlay; + AVFrame *output; + VAProcPipelineParameterBuffer params, subpic_params; + VABlendState blend_state; // Blend State + VARectangle overlay_region, output_region; + int err; + + err = overlay_vaapi_build_filter_params(avctx); + if (err < 0) + return err; + + err = ff_framesync_get_frame(fs, 0, &input_main, 0); + if (err < 0) + return err; + err = ff_framesync_get_frame(fs, 1, &input_overlay, 0); + if (err < 0) + return err; + + av_log(avctx, AV_LOG_DEBUG, "Filter main: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(input_main->format), + input_main->width, input_main->height, input_main->pts); + + av_log(avctx, AV_LOG_DEBUG, "Filter overlay: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(input_overlay->format), + input_overlay->width, input_overlay->height, input_overlay->pts); + + if (vpp_ctx->va_context == VA_INVALID_ID) + return AVERROR(EINVAL); + + output = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!output) { + err = AVERROR(ENOMEM); + goto fail; + } + + err = av_frame_copy_props(output, input_main); + if (err < 0) + goto fail; + + err = ff_vaapi_vpp_init_params(avctx, &params, + input_main, output); + if (err < 0) + goto fail; + + overlay_region = (VARectangle) { + .x = ctx->overlay_ox, + .y = ctx->overlay_oy, + .width = ctx->overlay_ow ? ctx->overlay_ow : input_overlay->width, + .height = ctx->overlay_oh ? ctx->overlay_oh : input_overlay->height, + }; + + output_region = (VARectangle) { + .x = 0, + .y = 0, + .width = output->width, + .height = output->height, + }; + + if (overlay_region.x + overlay_region.width > input_main->width || + overlay_region.y + overlay_region.height > input_main->height) { + av_log(ctx, AV_LOG_WARNING, + "The overlay image exceeds the scope of the main image, " + "will crop the overlay image according based on the main image.\n"); + } + + params.filters = &vpp_ctx->filter_buffers[0]; + params.num_filters = vpp_ctx->nb_filter_buffers; + + params.output_region = &output_region; + params.output_background_color = VAAPI_VPP_BACKGROUND_BLACK; + + memcpy(&subpic_params, &params, sizeof(subpic_params)); + + blend_state.flags = VA_BLEND_GLOBAL_ALPHA; + blend_state.global_alpha = ctx->alpha; + subpic_params.blend_state = &blend_state; + + subpic_params.surface = (VASurfaceID)(uintptr_t)input_overlay->data[3]; + subpic_params.output_region = &overlay_region; + + err = overlay_vaapi_render_picture(avctx, &params, &subpic_params, output); + if (err < 0) + goto fail; + + av_log(avctx, AV_LOG_DEBUG, "Filter output: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(output->format), + output->width, output->height, output->pts); + + return ff_filter_frame(outlink, output); + +fail: + av_frame_free(&output); + return err; +} + +static int overlay_vaapi_init_framesync(AVFilterContext *avctx) +{ + OverlayVAAPIContext *ctx = avctx->priv; + int ret, i; + + ctx->fs.on_event = overlay_vaapi_blend; + ctx->fs.opaque = ctx; + ret = ff_framesync_init(&ctx->fs, avctx, avctx->nb_inputs); + if (ret < 0) + return ret; + + for (i = 0; i < avctx->nb_inputs; i++) { + FFFrameSyncIn *in = &ctx->fs.in[i]; + in->before = EXT_STOP; + in->after = EXT_INFINITY; + in->sync = i ? 1 : 2; + in->time_base = avctx->inputs[i]->time_base; + } + + return ff_framesync_configure(&ctx->fs); +} + +static int overlay_vaapi_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + OverlayVAAPIContext *ctx = avctx->priv; + VAAPIVPPContext *vpp_ctx = avctx->priv; + int err; + + err = overlay_vaapi_init_framesync(avctx); + if (err < 0) + return err; + + vpp_ctx->output_width = avctx->inputs[0]->w; + vpp_ctx->output_height = avctx->inputs[0]->h; + + err = ff_vaapi_vpp_config_output(outlink); + if (err < 0) + return err; + + err = ff_framesync_init_dualinput(&ctx->fs, avctx); + if (err < 0) + return err; + + return ff_framesync_configure(&ctx->fs); +} + +static av_cold int overlay_vaapi_init(AVFilterContext *avctx) +{ + VAAPIVPPContext *vpp_ctx = avctx->priv; + + ff_vaapi_vpp_ctx_init(avctx); + vpp_ctx->output_format = AV_PIX_FMT_NONE; + + return 0; +} + +static int overlay_vaapi_activate(AVFilterContext *avctx) +{ + OverlayVAAPIContext *ctx = avctx->priv; + + return ff_framesync_activate(&ctx->fs); +} + +static av_cold void overlay_vaapi_uninit(AVFilterContext *avctx) +{ + OverlayVAAPIContext *ctx = avctx->priv; + + ff_framesync_uninit(&ctx->fs); +} + +#define OFFSET(x) offsetof(OverlayVAAPIContext, x) +#define FLAGS (AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM) +static const AVOption overlay_vaapi_options[] = { + { "x", "Overlay x position", + OFFSET(overlay_ox), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, + { "y", "Overlay y position", + OFFSET(overlay_oy), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, + { "w", "Overlay width", + OFFSET(overlay_ow), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, + { "h", "Overlay height", + OFFSET(overlay_oh), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, + { "alpha", "Overlay global alpha", + OFFSET(alpha), AV_OPT_TYPE_FLOAT, { .dbl = 0.0}, 0.0, 1.0, .flags = FLAGS}, + { NULL }, +}; + +AVFILTER_DEFINE_CLASS(overlay_vaapi); + +static const AVFilterPad overlay_vaapi_inputs[] = { + { + .name = "main", + .type = AVMEDIA_TYPE_VIDEO, + .get_video_buffer = ff_default_get_video_buffer, + .config_props = &ff_vaapi_vpp_config_input, + }, + { + .name = "overlay", + .type = AVMEDIA_TYPE_VIDEO, + .get_video_buffer = ff_default_get_video_buffer, + }, + { NULL } +}; + +static const AVFilterPad overlay_vaapi_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = &overlay_vaapi_config_output, + }, + { NULL } +}; + +AVFilter ff_vf_overlay_vaapi = { + .name = "overlay_vaapi", + .description = NULL_IF_CONFIG_SMALL("Overlay one video on top of another"), + .priv_size = sizeof(OverlayVAAPIContext), + .priv_class = &overlay_vaapi_class, + .init = &overlay_vaapi_init, + .uninit = &overlay_vaapi_uninit, + .query_formats = &overlay_vaapi_query_formats, + .activate = &overlay_vaapi_activate, + .inputs = overlay_vaapi_inputs, + .outputs = overlay_vaapi_outputs, + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, +}; -- 2.17.1

3 4

[PATCH] libavfilter/vf_codecview: add block structure visualization
by Yongle Lin 19 Nov '21

19 Nov '21

example command line to visualize block decomposition: ./ffmpeg -export_side_data +venc_params -i input.webm -vf codecview=bs=true output.webm --- doc/filters.texi | 3 +++ libavcodec/vp9.c | 8 +++----- libavfilter/vf_codecview.c | 41 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 47 insertions(+), 5 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 84567dec16..db2c80b1e9 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7285,6 +7285,9 @@ backward predicted MVs of B-frames @item qp Display quantization parameters using the chroma planes. +@item bs +Display block structure using the luma plane. + @item mv_type, mvt Set motion vectors type to visualize. Includes MVs from all frames unless specified by @var{frame_type} option. diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c index fd0bab14a2..e700def70e 100644 --- a/libavcodec/vp9.c +++ b/libavcodec/vp9.c @@ -1501,10 +1501,8 @@ static int vp9_export_enc_params(VP9Context *s, VP9Frame *frame) AVVideoEncParams *par; unsigned int tile, nb_blocks = 0; - if (s->s.h.segmentation.enabled) { - for (tile = 0; tile < s->active_tile_cols; tile++) - nb_blocks += s->td[tile].nb_block_structure; - } + for (tile = 0; tile < s->active_tile_cols; tile++) + nb_blocks += s->td[tile].nb_block_structure; par = av_video_enc_params_create_side_data(frame->tf.f, AV_VIDEO_ENC_PARAMS_VP9, nb_blocks); @@ -1536,7 +1534,7 @@ static int vp9_export_enc_params(VP9Context *s, VP9Frame *frame) b->w = 1 << (3 + td->block_structure[block_tile].block_size_idx_x); b->h = 1 << (3 + td->block_structure[block_tile].block_size_idx_y); - if (s->s.h.segmentation.feat[seg_id].q_enabled) { + if (s->s.h.segmentation.enabled && s->s.h.segmentation.feat[seg_id].q_enabled) { b->delta_qp = s->s.h.segmentation.feat[seg_id].q_val; if (s->s.h.segmentation.absolute_vals) b->delta_qp -= par->qp; diff --git a/libavfilter/vf_codecview.c b/libavfilter/vf_codecview.c index 331bfba777..db06625d70 100644 --- a/libavfilter/vf_codecview.c +++ b/libavfilter/vf_codecview.c @@ -34,6 +34,7 @@ #include "libavutil/opt.h" #include "avfilter.h" #include "internal.h" +#include "libavutil/video_enc_params.h" #define MV_P_FOR (1<<0) #define MV_B_FOR (1<<1) @@ -51,6 +52,7 @@ typedef struct CodecViewContext { unsigned mv_type; int hsub, vsub; int qp; + int bs; } CodecViewContext; #define OFFSET(x) offsetof(CodecViewContext, x) @@ -63,6 +65,7 @@ static const AVOption codecview_options[] = { CONST("bf", "forward predicted MVs of B-frames", MV_B_FOR, "mv"), CONST("bb", "backward predicted MVs of B-frames", MV_B_BACK, "mv"), { "qp", NULL, OFFSET(qp), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, .flags = FLAGS }, + { "bs", "set block structure to visualize", OFFSET(bs), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, .flags = FLAGS }, { "mv_type", "set motion vectors type", OFFSET(mv_type), AV_OPT_TYPE_FLAGS, {.i64=0}, 0, INT_MAX, FLAGS, "mv_type" }, { "mvt", "set motion vectors type", OFFSET(mv_type), AV_OPT_TYPE_FLAGS, {.i64=0}, 0, INT_MAX, FLAGS, "mv_type" }, CONST("fp", "forward predicted MVs", MV_TYPE_FOR, "mv_type"), @@ -212,6 +215,30 @@ static void draw_arrow(uint8_t *buf, int sx, int sy, int ex, draw_line(buf, sx, sy, ex, ey, w, h, stride, color); } +static void draw_block_border(AVFrame *frame, AVVideoBlockParams *b) +{ + const int lzy = frame->linesize[0]; + uint8_t *py = frame->data[0] + b->src_y * lzy; + + for (int x = b->src_x; x < b->src_x + b->w; x++) { + if (x >= frame->width) + break; + py[x] = py[x] * 3 / 4; + } + for (int y = b->src_y; y < b->src_y + b->h; y++) { + if (y >= frame->height) + break; + py[b->src_x] = py[b->src_x] * 3 / 4; + py[b->src_x + b->w - 1] = py[b->src_x + b->w - 1] * 3 / 4; + py += lzy; + } + for (int x = b->src_x; x < b->src_x + b->w; x++) { + if (x >= frame->width) + break; + py[x] = py[x] * 3 / 4; + } +} + static int filter_frame(AVFilterLink *inlink, AVFrame *frame) { AVFilterContext *ctx = inlink->dst; @@ -242,6 +269,20 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) } } + if (s->bs) { + AVFrameSideData *sd = av_frame_get_side_data(frame, AV_FRAME_DATA_VIDEO_ENC_PARAMS); + if (sd) { + AVVideoEncParams *par = (AVVideoEncParams*)sd->data; + + if (par->nb_blocks) { + for (int i = 0; i < par->nb_blocks; i++) { + AVVideoBlockParams *b = av_video_enc_params_block(par, i); + draw_block_border(frame, b); + } + } + } + } + if (s->mv || s->mv_type) { AVFrameSideData *sd = av_frame_get_side_data(frame, AV_FRAME_DATA_MOTION_VECTORS); if (sd) { -- 2.27.0.212.ge8ba1cc988-goog

3 10

[PATCH] avformat/libsrt: rename streamid
by raghavendra 06 Nov '21

06 Nov '21

Rename streamid to avoid name clashes. Signed-off-by: raghavendra <raghavendra.rao(a)collabora.com> --- libavformat/libsrt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavformat/libsrt.c b/libavformat/libsrt.c index 4025b24976..a490a386e6 100644 --- a/libavformat/libsrt.c +++ b/libavformat/libsrt.c @@ -134,7 +134,7 @@ static const AVOption libsrt_options[] = { { "rcvbuf", "Receive buffer size (in bytes)", OFFSET(rcvbuf), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, .flags = D|E }, { "lossmaxttl", "Maximum possible packet reorder tolerance", OFFSET(lossmaxttl), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, .flags = D|E }, { "minversion", "The minimum SRT version that is required from the peer", OFFSET(minversion), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, .flags = D|E }, - { "streamid", "A string of up to 512 characters that an Initiator can pass to a Responder", OFFSET(streamid), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = D|E }, + { "srt_streamid", "A string of up to 512 characters that an Initiator can pass to a Responder", OFFSET(streamid), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = D|E }, { "smoother", "The type of Smoother used for the transmission for that socket", OFFSET(smoother), AV_OPT_TYPE_STRING, { .str = NULL }, .flags = D|E }, { "messageapi", "Enable message API", OFFSET(messageapi), AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, .flags = D|E }, { "transtype", "The transmission type for the socket", OFFSET(transtype), AV_OPT_TYPE_INT, { .i64 = SRTT_INVALID }, SRTT_LIVE, SRTT_INVALID, .flags = D|E, "transtype" }, @@ -599,7 +599,7 @@ static int libsrt_open(URLContext *h, const char *uri, int flags) if (av_find_info_tag(buf, sizeof(buf), "minversion", p)) { s->minversion = strtol(buf, NULL, 0); } - if (av_find_info_tag(buf, sizeof(buf), "streamid", p)) { + if (av_find_info_tag(buf, sizeof(buf), "srt_streamid", p)) { av_freep(&s->streamid); s->streamid = av_strdup(buf); if (!s->streamid) { -- 2.25.1

6 17

[PATCH] avcodec/qsv_enc: do not reuse enc_ctrl from previous frames
by Xu Guangxin 10 Sep '21

10 Sep '21

fixes #8857 If we do not clear the enc_ctrl, we will reuse previous frames' data like FrameType. --- libavcodec/qsvenc.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 2bd2a56227..94473c4eab 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1249,6 +1249,8 @@ static void clear_unused_frames(QSVEncContext *q) while (cur) { if (cur->used && !cur->surface.Data.Locked) { free_encoder_ctrl_payloads(&cur->enc_ctrl); + //do not reuse enc_ctrl from previous frame + memset(&cur->enc_ctrl, 0, sizeof(cur->enc_ctrl)); if (cur->frame->format == AV_PIX_FMT_QSV) { av_frame_unref(cur->frame); } -- 2.17.1

3 6

[PATCH 0/2] fix for seeking in HLS with TS/FMP4 media
by vectronic 25 Aug '21

25 Aug '21

I am resubmitting a patch which fixes the following two tickets: https://trac.ffmpeg.org/ticket/7359 https://trac.ffmpeg.org/ticket/7485 The patch consists of 2 parts. The first is necessary to fix the case of HLS seeking when the HLS package consists of ts files. The second fixes HLS seeking when the HLS package consists of fmp4 files. Note that the first can be applied without the second, and HLS seeking with fmp4 files will remain broken, but in a different manner. The first patch works because it relies on correct behaviour implemented in mpegts.c, the second patch mirrors this behaviour in mov.c. The issues can be demonstrated as follows: ./ffmpeg -ss 3 -i http://vectronic.io/hls_seek_issue/ts/in.m3u8 -t 2 out.mp4 ./ffmpeg -ss 3 -i http://vectronic.io/hls_seek_issue/fmp4/in.m3u8 -t 2 out.mp4 These both produce zero duration output files. With the first patch applied, the ts example produces a 2 second output file as expected. However the fmp4 example results in a conversion error. With the second patch applied, both the ts and fmp4 examples produce 2 second output files as expected. Patch 1: Change to logic in hls.c for seeking After performing a rough seek in hls.c hls_read_seek() to the nearest segment, the logic for hls_read_packet() was to discard packets until a DTS after the accurate seek point is discovered. This was done even if AVSEEK_FLAG_ANY was not specified in the initial seek request. The patch changes this so that if AVSEEK_FLAG_ANY has not been specified, it will instead return the first keyframe discovered after the rough seek point. Patch 2: Change to logic in mov.c for reading packets after an HLS seek Within hls.c when hls_read_seek() is called it resets the stream position as follows: /* Reset the pos, to let the mpegts demuxer know we've seeked. */ pls->pb.pos = 0; There is support for this in the mpegts handle_packets() code to check if the position has been reset and clear out any PES packets: if (avio_tell(s->pb) != ts->last_pos) { int i; av_log(ts->stream, AV_LOG_TRACE, "Skipping after seek\n"); /* seek detected, flush pes buffer */ This behaviour needed to be mirrored in the mov demuxer. In mov.c it now detects if the pos has been reset in mov_read_packet(). It clears fragments and indexes and searches for the next root i.e. the next fragment via mov_switch_root(). vectronic (2): avformat hls fix to seek logic avformat mov fix to detect if stream position has been reset libavformat/hls.c | 8 +++++--- libavformat/mov.c | 36 +++++++++++++++++++++++++++++++++--- 2 files changed, 38 insertions(+), 6 deletions(-) -- 2.24.2 (Apple Git-127)

4 5

[PATCH] avformat/dv: fix timestamps of audio packets in case of dropped corrupt audio frames
by Marton Balint 22 Aug '21

22 Aug '21

Fixes out of sync timestamps in ticket #8762. Signed-off-by: Marton Balint <cus(a)passwd.hu> --- libavformat/dv.c | 16 ++-------------- tests/ref/seek/lavf-dv | 18 +++++++++--------- 2 files changed, 11 insertions(+), 23 deletions(-) diff --git a/libavformat/dv.c b/libavformat/dv.c index 3e0d12c0e3..26a78139f5 100644 --- a/libavformat/dv.c +++ b/libavformat/dv.c @@ -49,7 +49,6 @@ struct DVDemuxContext { uint8_t audio_buf[4][8192]; int ach; int frames; - uint64_t abytes; }; static inline uint16_t dv_audio_12to16(uint16_t sample) @@ -258,7 +257,7 @@ static int dv_extract_audio_info(DVDemuxContext *c, const uint8_t *frame) c->ast[i] = avformat_new_stream(c->fctx, NULL); if (!c->ast[i]) break; - avpriv_set_pts_info(c->ast[i], 64, 1, 30000); + avpriv_set_pts_info(c->ast[i], 64, c->sys->time_base.num, c->sys->time_base.den); c->ast[i]->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; c->ast[i]->codecpar->codec_id = AV_CODEC_ID_PCM_S16LE; @@ -387,8 +386,7 @@ int avpriv_dv_produce_packet(DVDemuxContext *c, AVPacket *pkt, for (i = 0; i < c->ach; i++) { c->audio_pkt[i].pos = pos; c->audio_pkt[i].size = size; - c->audio_pkt[i].pts = c->abytes * 30000 * 8 / - c->ast[i]->codecpar->bit_rate; + c->audio_pkt[i].pts = (c->sys->height == 720) ? (c->frames & ~1) : c->frames; ppcm[i] = c->audio_buf[i]; } if (c->ach) @@ -401,10 +399,7 @@ int avpriv_dv_produce_packet(DVDemuxContext *c, AVPacket *pkt, c->audio_pkt[2].size = c->audio_pkt[3].size = 0; } else { c->audio_pkt[0].size = c->audio_pkt[1].size = 0; - c->abytes += size; } - } else { - c->abytes += size; } /* Now it's time to return video packet */ @@ -444,13 +439,6 @@ static int64_t dv_frame_offset(AVFormatContext *s, DVDemuxContext *c, void ff_dv_offset_reset(DVDemuxContext *c, int64_t frame_offset) { c->frames = frame_offset; - if (c->ach) { - if (c->sys) { - c->abytes = av_rescale_q(c->frames, c->sys->time_base, - (AVRational) { 8, c->ast[0]->codecpar->bit_rate }); - } else - av_log(c->fctx, AV_LOG_ERROR, "cannot adjust audio bytes\n"); - } c->audio_pkt[0].size = c->audio_pkt[1].size = 0; c->audio_pkt[2].size = c->audio_pkt[3].size = 0; } diff --git a/tests/ref/seek/lavf-dv b/tests/ref/seek/lavf-dv index 0000ff5abe..f63e4460be 100644 --- a/tests/ref/seek/lavf-dv +++ b/tests/ref/seek/lavf-dv @@ -7,9 +7,9 @@ ret: 0 st: 0 flags:0 ts: 0.800000 ret: 0 st: 0 flags:1 dts: 0.800000 pts: 0.800000 pos:2880000 size:144000 ret: 0 st: 0 flags:1 ts:-0.320000 ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 size:144000 -ret: 0 st: 1 flags:0 ts: 2.576667 +ret: 0 st: 1 flags:0 ts: 2.560000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 -ret: 0 st: 1 flags:1 ts: 1.470833 +ret: 0 st: 1 flags:1 ts: 1.480000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 ret: 0 st:-1 flags:0 ts: 0.365002 ret: 0 st: 0 flags:1 dts: 0.360000 pts: 0.360000 pos:1296000 size:144000 @@ -19,9 +19,9 @@ ret: 0 st: 0 flags:0 ts: 2.160000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 ret: 0 st: 0 flags:1 ts: 1.040000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 -ret: 0 st: 1 flags:0 ts:-0.058333 +ret: 0 st: 1 flags:0 ts:-0.040000 ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 size:144000 -ret: 0 st: 1 flags:1 ts: 2.835833 +ret: 0 st: 1 flags:1 ts: 2.840000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 ret: 0 st:-1 flags:0 ts: 1.730004 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 @@ -31,10 +31,10 @@ ret: 0 st: 0 flags:0 ts:-0.480000 ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 size:144000 ret: 0 st: 0 flags:1 ts: 2.400000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 -ret: 0 st: 1 flags:0 ts: 1.306667 -ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 -ret: 0 st: 1 flags:1 ts: 0.200833 +ret: 0 st: 1 flags:0 ts: 1.320000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 +ret: 0 st: 1 flags:1 ts: 0.200000 +ret: 0 st: 0 flags:1 dts: 0.200000 pts: 0.200000 pos: 720000 size:144000 ret: 0 st:-1 flags:0 ts:-0.904994 ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 size:144000 ret: 0 st:-1 flags:1 ts: 1.989173 @@ -43,9 +43,9 @@ ret: 0 st: 0 flags:0 ts: 0.880000 ret: 0 st: 0 flags:1 dts: 0.880000 pts: 0.880000 pos:3168000 size:144000 ret: 0 st: 0 flags:1 ts:-0.240000 ret: 0 st: 0 flags:1 dts: 0.000000 pts: 0.000000 pos: 0 size:144000 -ret: 0 st: 1 flags:0 ts: 2.671667 +ret: 0 st: 1 flags:0 ts: 2.680000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 -ret: 0 st: 1 flags:1 ts: 1.565833 +ret: 0 st: 1 flags:1 ts: 1.560000 ret: 0 st: 0 flags:1 dts: 0.960000 pts: 0.960000 pos:3456000 size:144000 ret: 0 st:-1 flags:0 ts: 0.460008 ret: 0 st: 0 flags:1 dts: 0.480000 pts: 0.480000 pos:1728000 size:144000 -- 2.26.2

4 23

[PATCH 00/20] Matroska muxer patches
by Andreas Rheinhardt 20 Aug '21

20 Aug '21

Hello, this patchset is about the Matroska muxer. It consists partially of patches from my earlier patchset [1] that is currently blocked by the webm_chunk muxer [2]. One focus of this patchset is about improving writing Cues; in particular, an assert that is triggered if the reserved space for writing Cues in front exceeds the needed space by one byte is fixed. Furthermore, the checks involved in writing SeekHeads were fixed. This goes beyond what was done in the earlier patchset. - Andreas [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2019-November/252578.html [2]: https://ffmpeg.org/pipermail/ffmpeg-devel/2019-December/254783.html Andreas Rheinhardt (20): avformat/matroskaenc: Fix ReferenceBlock timestamp avformat/matroskaenc: Improve writing Projection avformat/matroskaenc: Remove useless AVIOContext avformat/matroskaenc: Adapt documentation of put_ebml_num avformat/matroskaenc: Check return value of ff_isom_write_hvcc() avformat/matroskaenc: Check functions that can fail avformat/matroskaenc: Avoid allocations for SeekHead avformat/matroskaenc: Improve calculating EBML ID size avformat/matroskaenc: Check for failure when writing SeekHead avformat/matroskaenc: Avoid unnecessary seek avformat/matroskaenc: Cosmetics avformat/matroskaenc: Avoid allocation for Cues avformat/aviobuf: Add function to reset dynamic buffer avformat/matroskadec: Simplify writing Cues avformat/matroskaenc: Write level 1 elements in one go avformat/matroskaenc: Remove unused function parameter avformat/matroskaenc: Check that Cluster has been opened avformat/matroskaenc: Don't fail if reserved Cues space doesn't suffice avformat/matroskaenc: Fix edge case of writing Cues at the beginning avformat/matroskaenc: Avoid seek when writing Cues at the front doc/muxers.texi | 4 +- libavformat/avio_internal.h | 7 + libavformat/aviobuf.c | 11 + libavformat/matroskaenc.c | 516 ++++++++++++--------------- tests/fate/matroska.mak | 2 +- tests/fate/wavpack.mak | 4 +- tests/ref/fate/aac-autobsf-adtstoasc | 4 +- tests/ref/fate/binsub-mksenc | 2 +- tests/ref/fate/rgb24-mkv | 4 +- tests/ref/lavf/mka | 4 +- tests/ref/lavf/mkv | 4 +- tests/ref/lavf/mkv_attachment | 4 +- tests/ref/seek/lavf-mkv | 44 +-- 13 files changed, 286 insertions(+), 324 deletions(-) -- 2.20.1

9 68

[PATCH 1/3] lavfi/asrc_sine: move sine table generation to a separate file.
by Nicolas George 19 Aug '21

19 Aug '21

Signed-off-by: Nicolas George <george(a)nsup.org> --- libavfilter/Makefile | 2 +- libavfilter/asrc_sine.c | 53 ++++----------------------------- libavfilter/intsine.c | 65 +++++++++++++++++++++++++++++++++++++++++ libavfilter/intsine.h | 44 ++++++++++++++++++++++++++++ 4 files changed, 115 insertions(+), 49 deletions(-) create mode 100644 libavfilter/intsine.c create mode 100644 libavfilter/intsine.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 5123540653..83d939f0b1 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -151,7 +151,7 @@ OBJS-$(CONFIG_ANULLSRC_FILTER) += asrc_anullsrc.o OBJS-$(CONFIG_FLITE_FILTER) += asrc_flite.o OBJS-$(CONFIG_HILBERT_FILTER) += asrc_hilbert.o OBJS-$(CONFIG_SINC_FILTER) += asrc_sinc.o -OBJS-$(CONFIG_SINE_FILTER) += asrc_sine.o +OBJS-$(CONFIG_SINE_FILTER) += asrc_sine.o intsine.o OBJS-$(CONFIG_ANULLSINK_FILTER) += asink_anullsink.o diff --git a/libavfilter/asrc_sine.c b/libavfilter/asrc_sine.c index 3a87210b4b..8fff1dda8b 100644 --- a/libavfilter/asrc_sine.c +++ b/libavfilter/asrc_sine.c @@ -27,6 +27,7 @@ #include "audio.h" #include "avfilter.h" #include "internal.h" +#include "intsine.h" typedef struct SineContext { const AVClass *class; @@ -81,50 +82,6 @@ static const AVOption sine_options[] = { AVFILTER_DEFINE_CLASS(sine); -#define LOG_PERIOD 15 -#define AMPLITUDE 4095 -#define AMPLITUDE_SHIFT 3 - -static void make_sin_table(int16_t *sin) -{ - unsigned half_pi = 1 << (LOG_PERIOD - 2); - unsigned ampls = AMPLITUDE << AMPLITUDE_SHIFT; - uint64_t unit2 = (uint64_t)(ampls * ampls) << 32; - unsigned step, i, c, s, k, new_k, n2; - - /* Principle: if u = exp(i*a1) and v = exp(i*a2), then - exp(i*(a1+a2)/2) = (u+v) / length(u+v) */ - sin[0] = 0; - sin[half_pi] = ampls; - for (step = half_pi; step > 1; step /= 2) { - /* k = (1 << 16) * amplitude / length(u+v) - In exact values, k is constant at a given step */ - k = 0x10000; - for (i = 0; i < half_pi / 2; i += step) { - s = sin[i] + sin[i + step]; - c = sin[half_pi - i] + sin[half_pi - i - step]; - n2 = s * s + c * c; - /* Newton's method to solve n² * k² = unit² */ - while (1) { - new_k = (k + unit2 / ((uint64_t)k * n2) + 1) >> 1; - if (k == new_k) - break; - k = new_k; - } - sin[i + step / 2] = (k * s + 0x7FFF) >> 16; - sin[half_pi - i - step / 2] = (k * c + 0x8000) >> 16; - } - } - /* Unshift amplitude */ - for (i = 0; i <= half_pi; i++) - sin[i] = (sin[i] + (1 << (AMPLITUDE_SHIFT - 1))) >> AMPLITUDE_SHIFT; - /* Use symmetries to fill the other three quarters */ - for (i = 0; i < half_pi; i++) - sin[half_pi * 2 - i] = sin[i]; - for (i = 0; i < 2 * half_pi; i++) - sin[i + 2 * half_pi] = -sin[i]; -} - static const char *const var_names[] = { "n", "pts", @@ -146,10 +103,10 @@ static av_cold int init(AVFilterContext *ctx) int ret; SineContext *sine = ctx->priv; - if (!(sine->sin = av_malloc(sizeof(*sine->sin) << LOG_PERIOD))) + if (!(sine->sin = av_malloc(sizeof(*sine->sin) << SINE_LOG_PERIOD))) return AVERROR(ENOMEM); sine->dphi = ldexp(sine->frequency, 32) / sine->sample_rate + 0.5; - make_sin_table(sine->sin); + ff_make_sin_table(sine->sin); if (sine->beep_factor) { sine->beep_period = sine->sample_rate; @@ -244,10 +201,10 @@ static int request_frame(AVFilterLink *outlink) samples = (int16_t *)frame->data[0]; for (i = 0; i < nb_samples; i++) { - samples[i] = sine->sin[sine->phi >> (32 - LOG_PERIOD)]; + samples[i] = sine->sin[sine->phi >> (32 - SINE_LOG_PERIOD)]; sine->phi += sine->dphi; if (sine->beep_index < sine->beep_length) { - samples[i] += sine->sin[sine->phi_beep >> (32 - LOG_PERIOD)] << 1; + samples[i] += sine->sin[sine->phi_beep >> (32 - SINE_LOG_PERIOD)] << 1; sine->phi_beep += sine->dphi_beep; } if (++sine->beep_index == sine->beep_period) diff --git a/libavfilter/intsine.c b/libavfilter/intsine.c new file mode 100644 index 0000000000..ad99dc0147 --- /dev/null +++ b/libavfilter/intsine.c @@ -0,0 +1,65 @@ +/* + * Copyright (c) 2020 Nicolas George + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "intsine.h" + +#define AMPLITUDE_SHIFT 3 + +void ff_make_sin_table(int16_t *sin) +{ + unsigned half_pi = 1 << (SINE_LOG_PERIOD - 2); + unsigned ampls = SINE_AMPLITUDE << AMPLITUDE_SHIFT; + uint64_t unit2 = (uint64_t)(ampls * ampls) << 32; + unsigned step, i, c, s, k, new_k, n2; + + /* Principle: if u = exp(i*a1) and v = exp(i*a2), then + exp(i*(a1+a2)/2) = (u+v) / length(u+v) */ + sin[0] = 0; + sin[half_pi] = ampls; + for (step = half_pi; step > 1; step /= 2) { + /* k = (1 << 16) * amplitude / length(u+v) + In exact values, k is constant at a given step */ + k = 0x10000; + for (i = 0; i < half_pi / 2; i += step) { + s = sin[i] + sin[i + step]; + c = sin[half_pi - i] + sin[half_pi - i - step]; + n2 = s * s + c * c; + /* Newton's method to solve n² * k² = unit² */ + while (1) { + new_k = (k + unit2 / ((uint64_t)k * n2) + 1) >> 1; + if (k == new_k) + break; + k = new_k; + } + sin[i + step / 2] = (k * s + 0x7FFF) >> 16; + sin[half_pi - i - step / 2] = (k * c + 0x8000) >> 16; + } + } + /* Unshift amplitude */ + for (i = 0; i <= half_pi; i++) + sin[i] = (sin[i] + (1 << (AMPLITUDE_SHIFT - 1))) >> AMPLITUDE_SHIFT; + /* Use symmetries to fill the other three quarters */ + for (i = 0; i < half_pi; i++) + sin[half_pi * 2 - i] = sin[i]; + for (i = 0; i < 2 * half_pi; i++) + sin[i + 2 * half_pi] = -sin[i]; +} + + diff --git a/libavfilter/intsine.h b/libavfilter/intsine.h new file mode 100644 index 0000000000..0723a9f141 --- /dev/null +++ b/libavfilter/intsine.h @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2020 Nicolas George + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_INTSINE_H +#define AVFILTER_INTSINE_H + +#include <stdint.h> + +/** + * Make a sin() table with integer arithmetic. + * + * The provided pointer must point to an array of SINE_PERIOD. + * It will be filled with one period ([0,2π[) of sine values with amplitude + * SINE_AMPLITUDE. + * + * A good way to use this table is to use a phase variable phi of type + * uint32_t, with 0 for 0 and 1<<32 for 2π and access the table at + * sin[phi >> SINE_SHIFT_PHI]. + */ +void ff_make_sin_table(int16_t *sin); + +#define SINE_LOG_PERIOD 15 +#define SINE_PERIOD (1U<<SINE_LOG_PERIOD) +#define SINE_AMPLITUDE 4095 +#define SINE_SHIFT_PHI (32 - SINE_LOG_PERIOD) + +#endif -- 2.26.2

6 16

[PATCH] lavc/qsvenc: allows the SDK runtime to choose LowPower/non-LowPower modes
by Haihao Xiang 11 Aug '21

11 Aug '21

The SDK supports LowPower and non-LowPower modes, but some features are available only under one of the two modes. Currently non-LowPower mode is always chosen in FFmpeg if the mode is not set explicitly, which will result in some SDK errors if a feature is unavailable under non-LowPower mode. With this patch, the mode is unknown in FFmpeg if the mode is not set explicitly, the SDK is responsible for mode selection --- This is the new version for "lavc/qsvenc: let the SDK to choose the encoding mode by default" and the git commit log is updated only libavcodec/qsvenc.c | 6 ++++-- libavcodec/qsvenc.h | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 1ed8f5d973..cff96e59c9 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -510,7 +510,7 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) } } - if (q->low_power) { + if (q->low_power == 1) { #if QSV_HAVE_VDENC q->param.mfx.LowPower = MFX_CODINGOPTION_ON; #else @@ -519,7 +519,9 @@ static int init_video_param(AVCodecContext *avctx, QSVEncContext *q) q->low_power = 0; q->param.mfx.LowPower = MFX_CODINGOPTION_OFF; #endif - } else + } else if (q->low_power == -1) + q->param.mfx.LowPower = MFX_CODINGOPTION_UNKNOWN; + else q->param.mfx.LowPower = MFX_CODINGOPTION_OFF; q->param.mfx.CodecProfile = q->profile; diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 4f579d1db1..577775cc1a 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -96,7 +96,7 @@ { "adaptive_b", "Adaptive B-frame placement", OFFSET(qsv.adaptive_b), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "b_strategy", "Strategy to choose between I/P/B-frames", OFFSET(qsv.b_strategy), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ { "forced_idr", "Forcing I frames as IDR frames", OFFSET(qsv.forced_idr), AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, VE }, \ -{ "low_power", "enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.)", OFFSET(qsv.low_power), AV_OPT_TYPE_BOOL, { .i64 = 0}, 0, 1, VE},\ +{ "low_power", "enable low power mode(experimental: many limitations by mfx version, BRC modes, etc.)", OFFSET(qsv.low_power), AV_OPT_TYPE_BOOL, { .i64 = -1}, -1, 1, VE},\ extern const AVCodecHWConfigInternal *ff_qsv_enc_hw_configs[]; -- 2.25.1

3 3