October 2019 - ffmpeg-devel - lists.ffmpeg.org

[PATCH 1/3] avfilter: add audio soft clip filter
by Paul B Mahol 25 Nov '23

25 Nov '23

Signed-off-by: Paul B Mahol <onemda(a)gmail.com> --- doc/filters.texi | 27 +++++ libavfilter/Makefile | 1 + libavfilter/af_asoftclip.c | 217 +++++++++++++++++++++++++++++++++++++ libavfilter/allfilters.c | 1 + 4 files changed, 246 insertions(+) create mode 100644 libavfilter/af_asoftclip.c diff --git a/doc/filters.texi b/doc/filters.texi index 4dd1a5de85..465eeb4732 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -2104,6 +2104,33 @@ audio, the data is treated as if all the planes were concatenated. A list of Adler-32 checksums for each data plane. @end table +@section asoftclip +Apply audio soft clipping. + +Soft clipping is a type of distortion effect where the amplitude of a signal is saturated +along a smooth curve, rather than the abrupt shape of hard-clipping. + +This filter accepts the following options: + +@table @option +@item type +Set type of soft-clipping. + +It accepts the following values: +@table @option +@item tanh +@item atan +@item cubic +@item exp +@item alg +@item quintic +@item sin +@end table + +@item param +Set additional parameter which controls sigmoid function. +@end table + @anchor{astats} @section astats diff --git a/libavfilter/Makefile b/libavfilter/Makefile index fef6ec5c55..682df45ef5 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -80,6 +80,7 @@ OBJS-$(CONFIG_ASETRATE_FILTER) += af_asetrate.o OBJS-$(CONFIG_ASETTB_FILTER) += settb.o OBJS-$(CONFIG_ASHOWINFO_FILTER) += af_ashowinfo.o OBJS-$(CONFIG_ASIDEDATA_FILTER) += f_sidedata.o +OBJS-$(CONFIG_ASOFTCLIP_FILTER) += af_asoftclip.o OBJS-$(CONFIG_ASPLIT_FILTER) += split.o OBJS-$(CONFIG_ASTATS_FILTER) += af_astats.o OBJS-$(CONFIG_ASTREAMSELECT_FILTER) += f_streamselect.o framesync.o diff --git a/libavfilter/af_asoftclip.c b/libavfilter/af_asoftclip.c new file mode 100644 index 0000000000..239b1d6a6b --- /dev/null +++ b/libavfilter/af_asoftclip.c @@ -0,0 +1,217 @@ +/* + * Copyright (c) 2019 The FFmpeg Project + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/channel_layout.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "audio.h" +#include "formats.h" + +enum ASoftClipTypes { + ASC_TANH, + ASC_ATAN, + ASC_CUBIC, + ASC_EXP, + ASC_ALG, + ASC_QUINTIC, + ASC_SIN, + NB_TYPES, +}; + +typedef struct ASoftClipContext { + const AVClass *class; + + int type; + double param; + + void (*filter)(struct ASoftClipContext *s, void **dst, const void **src, + int nb_samples, int channels); +} ASoftClipContext; + +#define OFFSET(x) offsetof(ASoftClipContext, x) +#define A AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption asoftclip_options[] = { + { "type", "set softclip type", OFFSET(type), AV_OPT_TYPE_INT, {.i64=0}, 0, NB_TYPES-1, A, "types" }, + { "tanh", NULL, 0, AV_OPT_TYPE_CONST, {.i64=ASC_TANH}, 0, 0, A, "types" }, + { "atan", NULL, 0, AV_OPT_TYPE_CONST, {.i64=ASC_ATAN}, 0, 0, A, "types" }, + { "cubic", NULL, 0, AV_OPT_TYPE_CONST, {.i64=ASC_CUBIC}, 0, 0, A, "types" }, + { "exp", NULL, 0, AV_OPT_TYPE_CONST, {.i64=ASC_EXP}, 0, 0, A, "types" }, + { "alg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=ASC_ALG}, 0, 0, A, "types" }, + { "quintic", NULL, 0, AV_OPT_TYPE_CONST, {.i64=ASC_QUINTIC},0, 0, A, "types" }, + { "sin", NULL, 0, AV_OPT_TYPE_CONST, {.i64=ASC_SIN}, 0, 0, A, "types" }, + { "param", "set softclip parameter", OFFSET(param), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.01, 3, A }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(asoftclip); + +static int query_formats(AVFilterContext *ctx) +{ + AVFilterFormats *formats = NULL; + AVFilterChannelLayouts *layouts = NULL; + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_DBLP, + AV_SAMPLE_FMT_NONE + }; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + ret = ff_set_common_formats(ctx, formats); + if (ret < 0) + return ret; + + layouts = ff_all_channel_counts(); + if (!layouts) + return AVERROR(ENOMEM); + + ret = ff_set_common_channel_layouts(ctx, layouts); + if (ret < 0) + return ret; + + formats = ff_all_samplerates(); + return ff_set_common_samplerates(ctx, formats); +} + +#define SQR(x) ((x) * (x)) + +static void filter_dblp(ASoftClipContext *s, + void **dptr, const void **sptr, + int nb_samples, int channels) +{ + double param = s->param; + int n, c; + + for (c = 0; c < channels; c++) { + const double *src = sptr[c]; + double *dst = dptr[c]; + + switch (s->type) { + case ASC_TANH: + for (n = 0; n < nb_samples; n++) { + dst[n] = tanh(src[n] * param); + } + break; + case ASC_ATAN: + for (n = 0; n < nb_samples; n++) + dst[n] = 2. / M_PI * atan(src[n] * param); + break; + case ASC_CUBIC: + for (n = 0; n < nb_samples; n++) { + if (FFABS(src[n]) >= 1.5) + dst[n] = FFSIGN(src[n]); + else + dst[n] = src[n] - 0.1481 * pow(src[n], 3.0f); + } + break; + case ASC_EXP: + for (n = 0; n < nb_samples; n++) + dst[n] = 2. / (1. + exp(-2. * src[n])) - 1.; + break; + case ASC_ALG: + for (n = 0; n < nb_samples; n++) + dst[n] = src[n] / (sqrt(param + src[n] * src[n])); + break; + case ASC_QUINTIC: + for (n = 0; n < nb_samples; n++) { + if (FFABS(src[n]) >= 1.25) + dst[n] = FFSIGN(src[n]); + else + dst[n] = src[n] - 0.08192 * pow(src[n], 5.); + } + break; + case ASC_SIN: + for (n = 0; n < nb_samples; n++) { + if (FFABS(src[n]) >= M_PI_2) + dst[n] = FFSIGN(src[n]); + else + dst[n] = sin(src[n]); + } + break; + } + } +} + +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + ASoftClipContext *s = ctx->priv; + + s->filter = filter_dblp; + + return 0; +} + +static int filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + ASoftClipContext *s = ctx->priv; + AVFrame *out; + + if (av_frame_is_writable(in)) { + out = in; + } else { + out = ff_get_audio_buffer(outlink, in->nb_samples); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); + } + + s->filter(s, (void **)out->extended_data, (const void **)in->extended_data, + in->nb_samples, in->channels); + + if (out != in) + av_frame_free(&in); + + return ff_filter_frame(outlink, out); +} + +static const AVFilterPad inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .filter_frame = filter_frame, + .config_props = config_input, + }, + { NULL } +}; + +static const AVFilterPad outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +AVFilter ff_af_asoftclip = { + .name = "asoftclip", + .description = NULL_IF_CONFIG_SMALL("Audio Soft Clipper."), + .query_formats = query_formats, + .priv_size = sizeof(ASoftClipContext), + .priv_class = &asoftclip_class, + .inputs = inputs, + .outputs = outputs, +}; diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index c51ae0f3c7..4d3039d6ba 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -72,6 +72,7 @@ extern AVFilter ff_af_asetrate; extern AVFilter ff_af_asettb; extern AVFilter ff_af_ashowinfo; extern AVFilter ff_af_asidedata; +extern AVFilter ff_af_asoftclip; extern AVFilter ff_af_asplit; extern AVFilter ff_af_astats; extern AVFilter ff_af_astreamselect; -- 2.17.1

6 47

[PATCH] avcodec/arm/sbcenc: save callee preserved vfp registers
by James Cowgill 12 Sep '22

12 Sep '22

When compiling FFmpeg with GCC-9, some very random segfaults were observed in code which had previously called down into the SBC encoder NEON assembly routines. This was caused by these functions clobbering some of the vfp callee saved registers (d8 - d15 aka q4 - q7). GCC was using these registers to save local variables, but after these functions returned, they would contain garbage. Fix by saving the relevant registers on the stack in the affected functions. Signed-off-by: James Cowgill <jcowgill(a)debian.org> --- libavcodec/arm/sbcdsp_neon.S | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavcodec/arm/sbcdsp_neon.S b/libavcodec/arm/sbcdsp_neon.S index d83d21d202..aa03800096 100644 --- a/libavcodec/arm/sbcdsp_neon.S +++ b/libavcodec/arm/sbcdsp_neon.S @@ -38,6 +38,8 @@ function ff_sbc_analyze_4_neon, export=1 /* TODO: merge even and odd cases (or even merge all four calls to this * function) in order to have only aligned reads from 'in' array * and reduce number of load instructions */ + vpush {d8-d11} + vld1.16 {d4, d5}, [r0, :64]! vld1.16 {d8, d9}, [r2, :128]! @@ -84,6 +86,7 @@ function ff_sbc_analyze_4_neon, export=1 vst1.32 {d0, d1}, [r1, :128] + vpop {d8-d11} bx lr endfunc @@ -91,6 +94,8 @@ function ff_sbc_analyze_8_neon, export=1 /* TODO: merge even and odd cases (or even merge all four calls to this * function) in order to have only aligned reads from 'in' array * and reduce number of load instructions */ + vpush {d8-d15} + vld1.16 {d4, d5}, [r0, :64]! vld1.16 {d8, d9}, [r2, :128]! @@ -188,6 +193,7 @@ function ff_sbc_analyze_8_neon, export=1 vst1.32 {d0, d1, d2, d3}, [r1, :128] + vpop {d8-d15} bx lr endfunc -- 2.22.0

3 3

[PATCH] avcodec/texturedspenc: Fix indexing in color distribution determination
by Michael Niedermayer 08 Jun '22

08 Jun '22

Fixes CID1396405 Untested except fate which does not test this, please someone test this Signed-off-by: Michael Niedermayer <michael(a)niedermayer.cc> --- libavcodec/texturedspenc.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libavcodec/texturedspenc.c b/libavcodec/texturedspenc.c index 8b2863033b..5171c69d82 100644 --- a/libavcodec/texturedspenc.c +++ b/libavcodec/texturedspenc.c @@ -255,11 +255,11 @@ static void optimize_colors(const uint8_t *block, ptrdiff_t stride, muv = minv = maxv = bp[0]; for (y = 0; y < 4; y++) { - for (x = 4; x < 4; x += 4) { + for (x = 0; x < 4; x++) { muv += bp[x * 4 + y * stride]; - if (bp[x] < minv) + if (bp[x * 4 + y * stride] < minv) minv = bp[x * 4 + y * stride]; - else if (bp[x] > maxv) + else if (bp[x * 4 + y * stride] > maxv) maxv = bp[x * 4 + y * stride]; } } -- 2.11.0

4 6

[PATCH] avcodec/ac3dec: Check expacc
by Michael Niedermayer 04 Mar '22

04 Mar '22

Signed-off-by: Michael Niedermayer <michael(a)niedermayer.cc> --- libavcodec/ac3dec.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/libavcodec/ac3dec.c b/libavcodec/ac3dec.c index a95c204..499971a 100644 --- a/libavcodec/ac3dec.c +++ b/libavcodec/ac3dec.c @@ -426,6 +426,10 @@ static int decode_exponents(AC3DecodeContext *s, group_size = exp_strategy + (exp_strategy == EXP_D45); for (grp = 0, i = 0; grp < ngrps; grp++) { expacc = get_bits(gbc, 7); + if (expacc >= 125) { + av_log(s->avctx, AV_LOG_ERROR, "expacc %d is out-of-range\n", expacc); + return AVERROR_INVALIDDATA; + } dexp[i++] = ungroup_3_in_7_bits_tab[expacc][0]; dexp[i++] = ungroup_3_in_7_bits_tab[expacc][1]; dexp[i++] = ungroup_3_in_7_bits_tab[expacc][2]; -- 2.10.2

3 4

[PATCH, v2] lavf/vf_deinterlace_vaapi: flush queued frame for field in DeinterlacingBob
by Linjie Fu 12 Jan '22

12 Jan '22

For DeinterlacingBob mode with rate=field, the frame number of output should equal 2x input total since only intra deinterlace is used. Currently for "backward_ref = 0, rate = field", extra_delay is introduced. Due to the async without flush, frame number of output is [expected_number - 2]. Specifically, if the input only has 1 frame, the output will be empty. Add deint_vaapi_request_frame for deinterlace_vaapi, send NULL frame to flush the queued frame. For 1 frame input in Bob mode with rate=field, before patch: 0 frame; after patch: 2 frames; ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.h264 -an -vf deinterlace_vaapi=mode=bob:rate=field -f null - Signed-off-by: Linjie Fu <linjie.fu(a)intel.com> --- libavfilter/vf_deinterlace_vaapi.c | 44 ++++++++++++++++++++++++++++++++------ 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/libavfilter/vf_deinterlace_vaapi.c b/libavfilter/vf_deinterlace_vaapi.c index 72d0349..fe77118 100644 --- a/libavfilter/vf_deinterlace_vaapi.c +++ b/libavfilter/vf_deinterlace_vaapi.c @@ -48,6 +48,9 @@ typedef struct DeintVAAPIContext { int queue_count; AVFrame *frame_queue[MAX_REFERENCES]; int extra_delay_for_timestamps; + + int eof; + int prev_pts; } DeintVAAPIContext; static const char *deint_vaapi_mode_name(int mode) @@ -190,9 +193,11 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) void *filter_params_addr = NULL; int err, i, field, current_frame_index; - av_log(avctx, AV_LOG_DEBUG, "Filter input: %s, %ux%u (%"PRId64").\n", - av_get_pix_fmt_name(input_frame->format), - input_frame->width, input_frame->height, input_frame->pts); + // NULL frame is used to flush the queue in field mode + if (input_frame) + av_log(avctx, AV_LOG_DEBUG, "Filter input: %s, %ux%u (%"PRId64").\n", + av_get_pix_fmt_name(input_frame->format), + input_frame->width, input_frame->height, input_frame->pts); if (ctx->queue_count < ctx->queue_depth) { ctx->frame_queue[ctx->queue_count++] = input_frame; @@ -210,6 +215,9 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) current_frame_index = ctx->pipeline_caps.num_forward_references; input_frame = ctx->frame_queue[current_frame_index]; + if (!input_frame) + return 0; + input_surface = (VASurfaceID)(uintptr_t)input_frame->data[3]; for (i = 0; i < ctx->pipeline_caps.num_forward_references; i++) forward_references[i] = (VASurfaceID)(uintptr_t) @@ -291,6 +299,8 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) if (ctx->field_rate == 2) { if (field == 0) output_frame->pts = 2 * input_frame->pts; + else if (ctx->eof) + output_frame->pts = 3 * input_frame->pts - ctx->prev_pts; else output_frame->pts = input_frame->pts + ctx->frame_queue[current_frame_index + 1]->pts; @@ -306,6 +316,8 @@ static int deint_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) break; } + ctx->prev_pts = input_frame->pts; + return err; fail: @@ -315,6 +327,25 @@ fail: return err; } +static int deint_vaapi_request_frame(AVFilterLink *link) +{ + AVFilterContext *avctx = link->src; + DeintVAAPIContext *ctx = avctx->priv; + int ret; + + if (ctx->eof) + return AVERROR_EOF; + + ret = ff_request_frame(link->src->inputs[0]); + if (ret == AVERROR_EOF && ctx->extra_delay_for_timestamps) { + ctx->eof = 1; + deint_vaapi_filter_frame(link->src->inputs[0], NULL); + } else if (ret < 0) + return ret; + + return 0; +} + static av_cold int deint_vaapi_init(AVFilterContext *avctx) { VAAPIVPPContext *vpp_ctx = avctx->priv; @@ -376,9 +407,10 @@ static const AVFilterPad deint_vaapi_inputs[] = { static const AVFilterPad deint_vaapi_outputs[] = { { - .name = "default", - .type = AVMEDIA_TYPE_VIDEO, - .config_props = &deint_vaapi_config_output, + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .request_frame = &deint_vaapi_request_frame, + .config_props = &deint_vaapi_config_output, }, { NULL } }; -- 2.7.4

3 4

[PATCH] set AVFrame decode_error_flags in case of decoding error by h264dec
by Amir Pauker 31 May '21

31 May '21

set AVFrame decode_error_flags in case h->slice_ctx->er.error_occurred is set after the call to ff_h264_execute_decode_slices. This allows the user to detect concealed decoding errors in the call to avcodec_receive_frame Signed-off-by: Amir Pauker <amir(a)livelyvideo.tv> --- libavcodec/error_resilience.c | 2 ++ libavcodec/h264dec.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/libavcodec/error_resilience.c b/libavcodec/error_resilience.c index 35d0c60..ca22871 100644 --- a/libavcodec/error_resilience.c +++ b/libavcodec/error_resilience.c @@ -1121,6 +1121,8 @@ void ff_er_frame_end(ERContext *s) av_log(s->avctx, AV_LOG_INFO, "concealing %d DC, %d AC, %d MV errors in %c frame\n", dc_error, ac_error, mv_error, av_get_picture_type_char(s->cur_pic.f->pict_type)); + s->cur_pic.f->decode_error_flags |= FF_DECODE_ERROR_CONCEALMENT_ACTIVE; + is_intra_likely = is_intra_more_likely(s); /* set unknown mb-type to most likely */ diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index 837c3b7..8d1bd16 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -761,6 +761,11 @@ static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size) if (ret < 0 && (h->avctx->err_recognition & AV_EF_EXPLODE)) goto end; + // set decode_error_flags to allow users to detect concealed decoding errors + if ((ret < 0 || h->slice_ctx->er.error_occurred) && h->cur_pic_ptr) { + h->cur_pic_ptr->f->decode_error_flags |= FF_DECODE_ERROR_DECODE_SLICES; + } + ret = 0; end: -- 2.1.4

5 5

[PATCH v2 01/15] avfilter/vf_xbr: Fix left shift of negative number
by Andreas Rheinhardt 26 Mar '21

26 Mar '21

Affected every usage of vf_xbr, e.g. the FATE-tests filter-2xbr, filter-3xbr, filter-4xbr. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt(a)gmail.com> --- libavfilter/vf_xbr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_xbr.c b/libavfilter/vf_xbr.c index 2c71871d22..87810c88f1 100644 --- a/libavfilter/vf_xbr.c +++ b/libavfilter/vf_xbr.c @@ -395,7 +395,7 @@ static int init(AVFilterContext *ctx) int startg = FFMAX3(-bg, -rg, 0); int endg = FFMIN3(255-bg, 255-rg, 255); uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000); - c = bg + (rg<<16) + 0x010101 * startg; + c = bg + rg * (1 << 16) + 0x010101 * startg; for (g = startg; g <= endg; g++) { s->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v; c+= 0x010101; -- 2.20.1

4 28

[PATCH] lavd/dshow: Add namespace prefix to global functions
by Mark Thompson 21 Jan '21

21 Jan '21

--- Not tested at all. libavdevice/dshow.c | 6 +++--- libavdevice/dshow_capture.h | 4 ++-- libavdevice/dshow_crossbar.c | 8 ++++---- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/libavdevice/dshow.c b/libavdevice/dshow.c index de910c0994..6f613f31b0 100644 --- a/libavdevice/dshow.c +++ b/libavdevice/dshow.c @@ -509,7 +509,7 @@ end: * Pops up a user dialog allowing them to adjust properties for the given filter, if possible. */ void -dshow_show_filter_properties(IBaseFilter *device_filter, AVFormatContext *avctx) { +ff_dshow_show_filter_properties(IBaseFilter *device_filter, AVFormatContext *avctx) { ISpecifyPropertyPages *property_pages = NULL; IUnknown *device_filter_iunknown = NULL; HRESULT hr; @@ -581,7 +581,7 @@ dshow_cycle_pins(AVFormatContext *avctx, enum dshowDeviceType devtype, int should_show_properties = (devtype == VideoDevice) ? ctx->show_video_device_dialog : ctx->show_audio_device_dialog; if (should_show_properties) - dshow_show_filter_properties(device_filter, avctx); + ff_dshow_show_filter_properties(device_filter, avctx); r = IBaseFilter_EnumPins(device_filter, &pins); if (r != S_OK) { @@ -886,7 +886,7 @@ dshow_open_device(AVFormatContext *avctx, ICreateDevEnum *devenum, goto error; } - r = dshow_try_setup_crossbar_options(graph_builder2, device_filter, devtype, avctx); + r = ff_dshow_try_setup_crossbar_options(graph_builder2, device_filter, devtype, avctx); if (r != S_OK) { av_log(avctx, AV_LOG_ERROR, "Could not setup CrossBar\n"); diff --git a/libavdevice/dshow_capture.h b/libavdevice/dshow_capture.h index 475d62ba99..79c004a87f 100644 --- a/libavdevice/dshow_capture.h +++ b/libavdevice/dshow_capture.h @@ -346,9 +346,9 @@ struct dshow_ctx { /***************************************************************************** * CrossBar ****************************************************************************/ -HRESULT dshow_try_setup_crossbar_options(ICaptureGraphBuilder2 *graph_builder2, +HRESULT ff_dshow_try_setup_crossbar_options(ICaptureGraphBuilder2 *graph_builder2, IBaseFilter *device_filter, enum dshowDeviceType devtype, AVFormatContext *avctx); -void dshow_show_filter_properties(IBaseFilter *pFilter, AVFormatContext *avctx); +void ff_dshow_show_filter_properties(IBaseFilter *pFilter, AVFormatContext *avctx); #endif /* AVDEVICE_DSHOW_CAPTURE_H */ diff --git a/libavdevice/dshow_crossbar.c b/libavdevice/dshow_crossbar.c index 95fb466f4e..2438683cde 100644 --- a/libavdevice/dshow_crossbar.c +++ b/libavdevice/dshow_crossbar.c @@ -137,7 +137,7 @@ setup_crossbar_options(IAMCrossbar *cross_bar, enum dshowDeviceType devtype, AVF * Given a fully constructed graph, check if there is a cross bar filter, and configure its pins if so. */ HRESULT -dshow_try_setup_crossbar_options(ICaptureGraphBuilder2 *graph_builder2, +ff_dshow_try_setup_crossbar_options(ICaptureGraphBuilder2 *graph_builder2, IBaseFilter *device_filter, enum dshowDeviceType devtype, AVFormatContext *avctx) { struct dshow_ctx *ctx = avctx->priv_data; @@ -163,7 +163,7 @@ dshow_try_setup_crossbar_options(ICaptureGraphBuilder2 *graph_builder2, hr = IAMCrossbar_QueryInterface(cross_bar, &IID_IBaseFilter, (void **) &cross_bar_base_filter); if (hr != S_OK) goto end; - dshow_show_filter_properties(cross_bar_base_filter, avctx); + ff_dshow_show_filter_properties(cross_bar_base_filter, avctx); } if (devtype == VideoDevice && ctx->show_analog_tv_tuner_dialog) { @@ -173,7 +173,7 @@ dshow_try_setup_crossbar_options(ICaptureGraphBuilder2 *graph_builder2, hr = IAMCrossbar_QueryInterface(tv_tuner_filter, &IID_IBaseFilter, (void **) &tv_tuner_base_filter); if (hr != S_OK) goto end; - dshow_show_filter_properties(tv_tuner_base_filter, avctx); + ff_dshow_show_filter_properties(tv_tuner_base_filter, avctx); } else { av_log(avctx, AV_LOG_WARNING, "unable to find a tv tuner to display dialog for!"); } @@ -185,7 +185,7 @@ dshow_try_setup_crossbar_options(ICaptureGraphBuilder2 *graph_builder2, hr = IAMCrossbar_QueryInterface(tv_audio_filter, &IID_IBaseFilter, (void **) &tv_audio_base_filter); if (hr != S_OK) goto end; - dshow_show_filter_properties(tv_audio_base_filter, avctx); + ff_dshow_show_filter_properties(tv_audio_base_filter, avctx); } else { av_log(avctx, AV_LOG_WARNING, "unable to find a tv audio tuner to display dialog for!"); } -- 2.18.0

2 2

[RFC PATCH] avfilter/fastdeint: import simple cpu-optimized deinterlacing algorithms from VLC
by Aman Gupta 02 Jan '21

02 Jan '21

From: Aman Gupta <aman(a)tmm1.net> These are simple algorithms which can be run efficiently on low powered devices to produce deinteraced images. Signed-off-by: Aman Gupta <aman(a)tmm1.net> --- doc/filters.texi | 27 ++ libavfilter/Makefile | 1 + libavfilter/aarch64/Makefile | 1 + libavfilter/aarch64/merge_neon.S | 98 ++++++ libavfilter/allfilters.c | 1 + libavfilter/arm/Makefile | 3 + libavfilter/arm/merge_armv6.S | 70 ++++ libavfilter/arm/merge_neon.S | 109 ++++++ libavfilter/vf_fastdeint.c | 588 +++++++++++++++++++++++++++++++ 9 files changed, 898 insertions(+) create mode 100644 libavfilter/aarch64/merge_neon.S create mode 100644 libavfilter/arm/Makefile create mode 100644 libavfilter/arm/merge_armv6.S create mode 100644 libavfilter/arm/merge_neon.S create mode 100644 libavfilter/vf_fastdeint.c diff --git a/doc/filters.texi b/doc/filters.texi index 6c81e1da40..55d9adeb81 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -9796,6 +9796,33 @@ fade=t=in:st=5.5:d=0.5 @end itemize +@section fastdeint +Fast deinterlacing algorithms. + +@table @option +@item mode +Deinterlacing algorithm to use. + +It accepts the following values: +@table @samp +@item discard +Discard bottom frame. + +@item mean +Half resolution blender. + +@item blend +Full resolution blender. + +@item bob +Bob doubler. + +@item linear +Bob doubler with linear interpolation. +@end table + +@end table + @section fftdnoiz Denoise frames using 3D FFT (frequency domain filtering). diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 3ef4191d9a..a2b3566ec0 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -234,6 +234,7 @@ OBJS-$(CONFIG_EROSION_OPENCL_FILTER) += vf_neighbor_opencl.o opencl.o \ opencl/neighbor.o OBJS-$(CONFIG_EXTRACTPLANES_FILTER) += vf_extractplanes.o OBJS-$(CONFIG_FADE_FILTER) += vf_fade.o +OBJS-$(CONFIG_FASTDEINT_FILTER) += vf_fastdeint.o OBJS-$(CONFIG_FFTDNOIZ_FILTER) += vf_fftdnoiz.o OBJS-$(CONFIG_FFTFILT_FILTER) += vf_fftfilt.o OBJS-$(CONFIG_FIELD_FILTER) += vf_field.o diff --git a/libavfilter/aarch64/Makefile b/libavfilter/aarch64/Makefile index b58daa3a3f..2b0ad92893 100644 --- a/libavfilter/aarch64/Makefile +++ b/libavfilter/aarch64/Makefile @@ -1,3 +1,4 @@ OBJS-$(CONFIG_NLMEANS_FILTER) += aarch64/vf_nlmeans_init.o +NEON-OBJS-$(CONFIG_FASTDEINT_FILTER) += aarch64/merge_neon.o NEON-OBJS-$(CONFIG_NLMEANS_FILTER) += aarch64/vf_nlmeans_neon.o diff --git a/libavfilter/aarch64/merge_neon.S b/libavfilter/aarch64/merge_neon.S new file mode 100644 index 0000000000..62377331a4 --- /dev/null +++ b/libavfilter/aarch64/merge_neon.S @@ -0,0 +1,98 @@ +/* + * Copyright (c) 2009-2016 Rémi Denis-Courmont, Janne Grunau, VLC authors + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/aarch64/asm.S" + +#define dest x0 +#define src1 x1 +#define src2 x2 +#define size x3 + + .align 2 + // NOTE: Offset and pitch must be multiple of 16-bytes. +function ff_merge8_neon, export=1 + ands x5, size, #~63 + b.eq 2f + mov x10, #64 + add x11, src1, #32 + add x12, src2, #32 +1: + ld1 {v0.16b,v1.16b}, [src1], x10 + ld1 {v4.16b,v5.16b}, [src2], x10 + ld1 {v2.16b,v3.16b}, [x11], x10 + uhadd v0.16b, v0.16b, v4.16b + ld1 {v6.16b,v7.16b}, [x12], x10 + subs x5, x5, #64 + uhadd v1.16b, v1.16b, v5.16b + uhadd v2.16b, v2.16b, v6.16b + uhadd v3.16b, v3.16b, v7.16b + st1 {v0.16b,v1.16b}, [dest], #32 + st1 {v2.16b,v3.16b}, [dest], #32 + b.gt 1b +2: + tbz size, #5, 3f + ld1 {v0.16b,v1.16b}, [src1], #32 + ld1 {v4.16b,v5.16b}, [src2], #32 + uhadd v0.16b, v0.16b, v4.16b + uhadd v1.16b, v1.16b, v5.16b + st1 {v0.16b,v1.16b}, [dest], #32 +3: + tbz size, #4, 4f + ld1 {v0.16b}, [src1] + ld1 {v4.16b}, [src2] + uhadd v0.16b, v0.16b, v4.16b + st1 {v0.16b}, [dest] +4: + ret +endfunc + + .align 2 +function ff_merge16_neon, export=1 + ands x5, size, #~63 + b.eq 2f +1: + ld1 {v0.8h,v1.8h}, [src1], #32 + ld1 {v4.8h,v5.8h}, [src2], #32 + ld1 {v2.8h,v3.8h}, [src1], #32 + uhadd v0.8h, v0.8h, v4.8h + ld1 {v6.8h,v7.8h}, [src2], #32 + uhadd v1.8h, v1.8h, v5.8h + uhadd v2.8h, v2.8h, v6.8h + uhadd v3.8h, v3.8h, v7.8h + st1 {v0.8h,v1.8h}, [dest], #32 + st1 {v2.8h,v3.8h}, [dest], #32 + subs x5, x5, #64 + b.gt 1b +2: + tbz size, #5, 3f + ld1 {v0.8h,v1.8h}, [src1], #32 + ld1 {v4.8h,v5.8h}, [src2], #32 + uhadd v0.8h, v0.8h, v4.8h + uhadd v1.8h, v1.8h, v5.8h + st1 {v0.8h,v1.8h}, [dest], #32 +3: + tbz size, #4, 4f + ld1 {v0.8h}, [src1] + ld1 {v4.8h}, [src2] + uhadd v0.8h, v0.8h,v4.8h + st1 {v0.8h}, [dest] +4: + ret +endfunc diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index b675c688ee..6631af2ffe 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -219,6 +219,7 @@ extern AVFilter ff_vf_erosion; extern AVFilter ff_vf_erosion_opencl; extern AVFilter ff_vf_extractplanes; extern AVFilter ff_vf_fade; +extern AVFilter ff_vf_fastdeint; extern AVFilter ff_vf_fftdnoiz; extern AVFilter ff_vf_fftfilt; extern AVFilter ff_vf_field; diff --git a/libavfilter/arm/Makefile b/libavfilter/arm/Makefile new file mode 100644 index 0000000000..c92d62fac9 --- /dev/null +++ b/libavfilter/arm/Makefile @@ -0,0 +1,3 @@ +ARMV6-OBJS-$(CONFIG_FASTDEINT_FILTER) += arm/merge_armv6.o + +NEON-OBJS-$(CONFIG_FASTDEINT_FILTER) += arm/merge_neon.o diff --git a/libavfilter/arm/merge_armv6.S b/libavfilter/arm/merge_armv6.S new file mode 100644 index 0000000000..9b551c2c6c --- /dev/null +++ b/libavfilter/arm/merge_armv6.S @@ -0,0 +1,70 @@ +/* + * Copyright (c) 2009-2012 Rémi Denis-Courmont, VLC authors + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/arm/asm.S" + +#define dest r0 +#define src1 r1 +#define src2 r2 +#define size r3 + + .align 2 +function ff_merge8_armv6, export=1 + push {r4-r9,lr} +1: + pld [src1, #64] + ldm src1!, {r4-r5} + pld [src2, #64] + ldm src2!, {r8-r9} + subs size, size, #16 + uhadd8 r4, r4, r8 + ldm src1!, {r6-r7} + uhadd8 r5, r5, r9 + ldm src2!, {ip,lr} + uhadd8 r6, r6, ip + stm dest!, {r4-r5} + uhadd8 r7, r7, lr + stm dest!, {r6-r7} + it eq + popeq {r4-r9,pc} + b 1b +endfunc + + .align 2 +function ff_merge16_armv6, export=1 + push {r4-r9,lr} +1: + pld [src1, #64] + ldm src1!, {r4-r5} + pld [src2, #64] + ldm src2!, {r8-r9} + subs size, size, #16 + uhadd16 r4, r4, r8 + ldm src1!, {r6-r7} + uhadd16 r5, r5, r9 + ldm src2!, {ip,lr} + uhadd16 r6, r6, ip + stm dest!, {r4-r5} + uhadd16 r7, r7, lr + stm dest!, {r6-r7} + it eq + popeq {r4-r9,pc} + b 1b +endfunc \ No newline at end of file diff --git a/libavfilter/arm/merge_neon.S b/libavfilter/arm/merge_neon.S new file mode 100644 index 0000000000..ae36cf3ca9 --- /dev/null +++ b/libavfilter/arm/merge_neon.S @@ -0,0 +1,109 @@ +/* + * Copyright (c) 2009-2012 Rémi Denis-Courmont, VLC authors + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/arm/asm.S" + +#define dest r0 +#define src1 r1 +#define src2 r2 +#define size r3 + + .align 2 + @ NOTE: Offset and pitch must be multiple of 16-bytes. +function ff_merge8_neon, export=1 + cmp size, #64 + blo 2f +1: + pld [src1, #64] + vld1.u8 {q0-q1}, [src1,:128]! + pld [src2, #64] + vld1.u8 {q8-q9}, [src2,:128]! + vhadd.u8 q0, q0, q8 + sub size, size, #64 + vld1.u8 {q2-q3}, [src1,:128]! + vhadd.u8 q1, q1, q9 + vld1.u8 {q10-q11}, [src2,:128]! + vhadd.u8 q2, q2, q10 + cmp size, #64 + vhadd.u8 q3, q3, q11 + vst1.u8 {q0-q1}, [dest,:128]! + vst1.u8 {q2-q3}, [dest,:128]! + bhs 1b +2: + cmp size, #32 + blo 3f + vld1.u8 {q0-q1}, [src1,:128]! + sub size, size, #32 + vld1.u8 {q8-q9}, [src2,:128]! + vhadd.u8 q0, q0, q8 + vhadd.u8 q1, q1, q9 + vst1.u8 {q0-q1}, [dest,:128]! +3: + cmp size, #16 + it lo + bxlo lr + vld1.u8 {q0}, [src1,:128]! + sub size, size, #16 + vld1.u8 {q8}, [src2,:128]! + vhadd.u8 q0, q0, q8 + vst1.u8 {q0}, [dest,:128]! + bx lr +endfunc + + .align 2 +function ff_merge16_neon, export=1 + cmp size, #64 + blo 2f +1: + pld [src1, #64] + vld1.u16 {q0-q1}, [src1,:128]! + pld [src2, #64] + vld1.u16 {q8-q9}, [src2,:128]! + vhadd.u16 q0, q0, q8 + sub size, size, #64 + vld1.u16 {q2-q3}, [src1,:128]! + vhadd.u16 q1, q1, q9 + vld1.u16 {q10-q11}, [src2,:128]! + vhadd.u16 q2, q2, q10 + cmp size, #64 + vhadd.u16 q3, q3, q11 + vst1.u16 {q0-q1}, [dest,:128]! + vst1.u16 {q2-q3}, [dest,:128]! + bhs 1b +2: + cmp size, #32 + blo 3f + vld1.u16 {q0-q1}, [src1,:128]! + sub size, size, #32 + vld1.u16 {q8-q9}, [src2,:128]! + vhadd.u16 q0, q0, q8 + vhadd.u16 q1, q1, q9 + vst1.u16 {q0-q1}, [dest,:128]! +3: + cmp size, #16 + it lo + bxlo lr + vld1.u16 {q0}, [src1,:128]! + sub size, size, #16 + vld1.u16 {q8}, [src2,:128]! + vhadd.u16 q0, q0, q8 + vst1.u16 {q0}, [dest,:128]! + bx lr +endfunc \ No newline at end of file diff --git a/libavfilter/vf_fastdeint.c b/libavfilter/vf_fastdeint.c new file mode 100644 index 0000000000..5ddd8be392 --- /dev/null +++ b/libavfilter/vf_fastdeint.c @@ -0,0 +1,588 @@ +/* + * Copyright (C) 2015 Aman Gupta <aman(a)tmm1.net> + * 2000-2011 VLC authors and VideoLAN + * + * Author: Sam Hocevar <sam(a)zoy.org> + * Damien Lucas <nitrox(a)videolan.org> + * Laurent Aimar <fenrir(a)videolan.org> + * Sigmund Augdal Helberg <sigmunau(a)videolan.org> + * + * These algorithms are derived from the VLC project's + * modules/video_filter/deinterlace/algo_basic.c + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/cpu.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/imgutils.h" +#include "libavutil/timestamp.h" +#include "avfilter.h" +#include "formats.h" +#include "internal.h" +#include "video.h" + +enum Mode { + MODE_DISCARD, + MODE_MEAN, + MODE_BLEND, + MODE_BOB, + MODE_LINEAR, + MODE_MAX, +}; + +typedef void (*merge_fn)(void *dst, const void *src1, const void *src2, size_t len); + +typedef struct FastDeintContext { + const AVClass *class; + merge_fn merge; + int merge_size; + int merge_aligned; + AVFrame *cur, *next; + enum Mode mode; + int eof; +} FastDeintContext; + +static void merge8_c(uint8_t *dst, const uint8_t *src1, const uint8_t *src2, size_t bytes) +{ + for (; bytes > 0; bytes--) + *dst++ = ( *src1++ + *src2++ ) >> 1; +} + +static void merge16_c(uint16_t *dst, const uint16_t *src1, const uint16_t *src2, size_t bytes) +{ + for (size_t words = bytes / 2; words > 0; words--) + *dst++ = ( *src1++ + *src2++ ) >> 1; +} + +static void merge8_unaligned(FastDeintContext *s, uint8_t *dst, const uint8_t *src1, const uint8_t *src2, size_t bytes) +{ + if (s->merge_aligned) { + size_t remainder = bytes % 16; + if (remainder > 0) { + merge8_c(dst, src1, src2, remainder); + bytes -= remainder; + dst += remainder; + src1 += remainder; + src2 += remainder; + } + } + s->merge(dst, src1, src2, bytes); +} + +static void merge16_unaligned(FastDeintContext *s, uint16_t *dst, const uint16_t *src1, const uint16_t *src2, size_t bytes) +{ + if (s->merge_aligned) { + size_t words = bytes / 2; + size_t remainder = words % 8; + if (remainder > 0) { + merge16_c(dst, src1, src2, remainder); + words -= remainder; + dst += remainder; + src1 += remainder; + src2 += remainder; + } + } + s->merge(dst, src1, src2, bytes); +} + +static void merge_unaligned(FastDeintContext *s, void *dst, const void *src1, const void *src2, size_t bytes) +{ + if (s->merge_size == 16) + merge16_unaligned(s, dst, src1, src2, bytes); + else + merge8_unaligned(s, dst, src1, src2, bytes); +} + +#if HAVE_SSE2_INLINE && defined(__x86_64__) +static void merge8_sse2(uint8_t *dst, const uint8_t *src1, const uint8_t *src2, size_t bytes) +{ + for(; bytes > 0 && ((uintptr_t)src1 & 15); bytes--) + *dst++ = ( *src1++ + *src2++ ) >> 1; + + for (; bytes >= 16; bytes -= 16) { + __asm__ __volatile__( "movdqu %2,%%xmm1;" + "pavgb %1, %%xmm1;" + "movdqu %%xmm1, %0" :"=m" (*dst): + "m" (*src1), + "m" (*src2) : "xmm1" ); + dst += 16; + src1 += 16; + src2 += 16; + } + + if (bytes > 0) { + merge8_c(dst, src1, src2, bytes); + } +} +static void merge16_sse2(uint16_t *dst, const uint16_t *src1, const uint16_t *src2, size_t bytes) +{ + size_t words = bytes / 2; + + for(; words > 0 && ((uintptr_t)src1 & 15); words--) + *dst++ = ( *src1++ + *src2++ ) >> 1; + + for (; words >= 8; words -= 8) { + __asm__ __volatile__( "movdqu %2,%%xmm1;" + "pavgw %1, %%xmm1;" + "movdqu %%xmm1, %0" :"=m" (*dst): + "m" (*src1), + "m" (*src2) : "xmm1" ); + dst += 8; + src1 += 8; + src2 += 8; + } + + if (words > 0) { + merge16_c(dst, src1, src2, words * 2); + } +} +#define merge8 merge8_sse2 +#define merge16 merge16_sse2 +#else +#define merge8 merge8_c +#define merge16 merge16_c +#endif + +static void render_image_single(FastDeintContext *s, AVFrame *out, AVFrame *frame) +{ + int i, planes_nb = 0; + enum Mode mode = s->mode; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(out->format); + + for (i = 0; i < desc->nb_components; i++) + planes_nb = FFMAX(planes_nb, desc->comp[i].plane + 1); + + for (i = 0; i < planes_nb; i++) { + int height, bwidth; + int dst_linesize, src_linesize; + const uint8_t *src; + uint8_t *dst; + + bwidth = av_image_get_linesize(out->format, out->width, i); + if (bwidth < 0) { + av_log(s, AV_LOG_ERROR, "av_image_get_linesize failed\n"); + return; + } + + height = out->height; + if (i == 1 || i == 2) { + height = FF_CEIL_RSHIFT(out->height, desc->log2_chroma_h); + } + + src = frame->data[i]; + dst = out->data[i]; + dst_linesize = out->linesize[i]; + src_linesize = frame->linesize[i]; + + if (mode == MODE_BLEND) { + // Copy first line + memcpy(dst, src, bwidth); + dst += dst_linesize; + height--; + } + + // Merge remaining lines + for (; height > 0; height--) { + if (mode == MODE_DISCARD) + memcpy(dst, src, bwidth); + else + merge_unaligned(s, dst, src, src + src_linesize, bwidth); + dst += dst_linesize; + src += src_linesize; + if (mode == MODE_MEAN || mode == MODE_DISCARD) { + src += src_linesize; + height--; + } + } + } + if (mode != MODE_DISCARD) + emms_c(); +} + +static void render_image_doubler(FastDeintContext *s, AVFrame *out, AVFrame *frame, int field) +{ + int i, planes_nb = 0; + enum Mode mode = s->mode; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(out->format); + + for (i = 0; i < desc->nb_components; i++) + planes_nb = FFMAX(planes_nb, desc->comp[i].plane + 1); + + for (i = 0; i < planes_nb; i++) { + int height, bwidth; + int dst_linesize, src_linesize; + const uint8_t *src; + uint8_t *dst; + + bwidth = av_image_get_linesize(out->format, out->width, i); + if (bwidth < 0) { + av_log(s, AV_LOG_ERROR, "av_image_get_linesize failed\n"); + return; + } + height = out->height; + if (i == 1 || i == 2) { + height = FF_CEIL_RSHIFT(out->height, desc->log2_chroma_h); + } + + src = frame->data[i]; + dst = out->data[i]; + src_linesize = frame->linesize[i]; + dst_linesize = out->linesize[i]; + + // For BOTTOM field we need to add the first line + if (field == 1) { + memcpy(dst, src, bwidth); + dst += dst_linesize; + src += src_linesize; + height--; + } + + height -= 2; + + for (; height > 0; height-=2) { + memcpy(dst, src, bwidth); + dst += dst_linesize; + + if (mode == MODE_LINEAR) + merge_unaligned(s, dst, src, src + 2 * src_linesize, bwidth); + else + memcpy(dst, src, bwidth); + dst += dst_linesize; + + src += src_linesize * 2; + } + + memcpy(dst, src, bwidth); + + // For TOP field we need to add the last line + if (field == 0) + { + dst += dst_linesize; + src += src_linesize; + memcpy(dst, src, bwidth); + } + } + if (mode == MODE_LINEAR) + emms_c(); +} + +static int filter_frame_single(AVFilterLink *link, AVFrame *frame) +{ + AVFilterContext *ctx = link->dst; + AVFrame *out; + FastDeintContext *s = ctx->priv; + + if (!frame->interlaced_frame) { + return ff_filter_frame(ctx->outputs[0], frame); + } + + out = ff_get_video_buffer(ctx->outputs[0], link->w, link->h); + if (!out) { + av_frame_free(&frame); + return AVERROR(ENOMEM); + } + + av_frame_copy_props(out, frame); + out->interlaced_frame = 0; + render_image_single(s, out, frame); + + av_frame_free(&frame); + return ff_filter_frame(ctx->outputs[0], out); +} + +static AVFrame *copy_frame(AVFilterLink *link, AVFrame *frame) +{ + AVFilterContext *ctx = link->dst; + AVFrame *out; + + if (frame->format == AV_PIX_FMT_VIDEOTOOLBOX) + out = av_frame_alloc(); + else + out = ff_get_video_buffer(ctx->outputs[0], link->w, link->h); + + if (!out) + return NULL; + + av_frame_copy_props(out, frame); + return out; +} + +static int filter_frame_double(AVFilterLink *link, AVFrame *in) +{ + AVFilterContext *ctx = link->dst; + FastDeintContext *s = ctx->priv; + AVFrame *frame, *out, *out2; + int tff, ret; + + s->cur = s->next; + s->next = in; + + if (!s->cur) { + return 0; + } + + frame = s->cur; + + if (!frame->interlaced_frame) { + if (frame->pts != AV_NOPTS_VALUE) + frame->pts *= 2; + s->cur = NULL; + return ff_filter_frame(ctx->outputs[0], frame); + } + + tff = frame->top_field_first; + out = copy_frame(link, frame); + if (!out) { + av_frame_free(&frame); + s->cur = NULL; + return AVERROR(ENOMEM); + } + + out->interlaced_frame = 0; + if (out->pts != AV_NOPTS_VALUE) + out->pts = out->pts * 2; + render_image_doubler(s, out, frame, !tff); + + ret = ff_filter_frame(ctx->outputs[0], out); + if (ret < 0) { + av_frame_free(&frame); + s->cur = NULL; + return ret; + } + + out2 = copy_frame(link, frame); + if (!out2) { + av_frame_free(&frame); + s->cur = NULL; + return AVERROR(ENOMEM); + } + + out2->interlaced_frame = 0; + av_frame_remove_side_data(out2, AV_FRAME_DATA_A53_CC); + if (out2->pts != AV_NOPTS_VALUE) { + out2->pts = frame->pts + s->next->pts; + } + render_image_doubler(s, out2, frame, tff); + + av_frame_free(&frame); + s->cur = NULL; + + return ff_filter_frame(ctx->outputs[0], out2); +} + +static int filter_frame(AVFilterLink *link, AVFrame *frame) +{ + AVFilterContext *ctx = link->dst; + FastDeintContext *s = ctx->priv; + + av_assert0(frame); + + if (s->mode == MODE_LINEAR || s->mode == MODE_BOB) { + return filter_frame_double(link, frame); + } else { + return filter_frame_single(link, frame); + } +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + FastDeintContext *s = ctx->priv; + av_frame_free(&s->cur); + av_frame_free(&s->next); +} + +static int query_formats(AVFilterContext *ctx) +{ + static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV422P, + AV_PIX_FMT_YUV444P, + AV_PIX_FMT_YUV410P, + AV_PIX_FMT_YUV411P, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_YUVJ420P, + AV_PIX_FMT_YUVJ422P, + AV_PIX_FMT_YUVJ444P, + AV_PIX_FMT_GRAY16, + AV_PIX_FMT_YUV440P, + AV_PIX_FMT_YUVJ440P, + AV_PIX_FMT_YUV420P9, + AV_PIX_FMT_YUV422P9, + AV_PIX_FMT_YUV444P9, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUV422P10, + AV_PIX_FMT_YUV444P10, + AV_PIX_FMT_YUV420P12, + AV_PIX_FMT_YUV422P12, + AV_PIX_FMT_YUV444P12, + AV_PIX_FMT_YUV420P14, + AV_PIX_FMT_YUV422P14, + AV_PIX_FMT_YUV444P14, + AV_PIX_FMT_YUV420P16, + AV_PIX_FMT_YUV422P16, + AV_PIX_FMT_YUV444P16, + AV_PIX_FMT_YUVA420P, + AV_PIX_FMT_YUVA422P, + AV_PIX_FMT_YUVA444P, + AV_PIX_FMT_GBRP, + AV_PIX_FMT_GBRP9, + AV_PIX_FMT_GBRP10, + AV_PIX_FMT_GBRP12, + AV_PIX_FMT_GBRP14, + AV_PIX_FMT_GBRP16, + AV_PIX_FMT_GBRAP, + AV_PIX_FMT_NONE + }; + + AVFilterFormats *fmts_list = ff_make_format_list(pix_fmts); + if (!fmts_list) + return AVERROR(ENOMEM); + return ff_set_common_formats(ctx, fmts_list); +} + +#if ARCH_ARM +#include "libavutil/arm/cpu.h" +#endif +#if ARCH_AARCH64 +#include "libavutil/aarch64/cpu.h" +#endif +#if ARCH_AARCH64 || ARCH_ARM +void ff_merge8_neon(uint8_t *dst, const uint8_t *src1, const uint8_t *src2, size_t bytes); +void ff_merge16_neon(uint16_t *dst, const uint16_t *src1, const uint16_t *src2, size_t bytes); +void ff_merge8_armv6(uint8_t *dst, const uint8_t *src1, const uint8_t *src2, size_t bytes); +void ff_merge16_armv6(uint16_t *dst, const uint16_t *src1, const uint16_t *src2, size_t bytes); +#endif + +static int config_props(AVFilterLink *link) +{ + AVFilterContext *ctx = link->src; + FastDeintContext *s = ctx->priv; + const AVPixFmtDescriptor *pix; +#if ARCH_AARCH64 || ARCH_ARM + int cpu_flags = av_get_cpu_flags(); +#endif + + link->w = link->src->inputs[0]->w; + link->h = link->src->inputs[0]->h; + link->time_base = link->src->inputs[0]->time_base; + link->frame_rate = link->src->inputs[0]->frame_rate; + link->sample_aspect_ratio = link->src->inputs[0]->sample_aspect_ratio; + + if (s->mode == MODE_MEAN || s->mode == MODE_DISCARD) { + link->h /= 2; + link->sample_aspect_ratio = av_mul_q(link->sample_aspect_ratio, av_make_q(1, 2)); + } + if (s->mode == MODE_LINEAR || s->mode == MODE_BOB) { + link->time_base = av_mul_q(link->time_base, av_make_q(1, 2)); + link->frame_rate = av_mul_q(link->frame_rate, av_make_q(2, 1)); + } + + pix = av_pix_fmt_desc_get(link->format); + s->merge_size = (pix->comp[0].depth > 8) ? 16 : 8; + s->merge = s->merge_size == 16 ? (merge_fn)merge16 : (merge_fn)merge8; + +#if ARCH_ARM + if (have_armv6(cpu_flags)) { + s->merge = s->merge_size == 16 ? (merge_fn)ff_merge16_armv6 : (merge_fn)ff_merge8_armv6; + s->merge_aligned = 1; + } +#endif +#if ARCH_AARCH64 || ARCH_ARM + if (have_neon(cpu_flags)) { + s->merge = s->merge_size == 16 ? (merge_fn)ff_merge16_neon : (merge_fn)ff_merge8_neon; + s->merge_aligned = 1; + } +#endif + + return 0; +} + +static int request_frame(AVFilterLink *link) +{ + AVFilterContext *ctx = link->src; + FastDeintContext *s = ctx->priv; + int ret; + + if (s->eof) + return AVERROR_EOF; + + ret = ff_request_frame(ctx->inputs[0]); + + if (ret == AVERROR_EOF && s->cur) { + AVFrame *next = av_frame_clone(s->next); + if (!next) + return AVERROR(ENOMEM); + + next->pts = s->next->pts * 2 - s->cur->pts; + filter_frame(ctx->inputs[0], next); + s->eof = 1; + } else if (ret < 0) { + return ret; + } + + return 0; +} + +#define OFFSET(x) offsetof(FastDeintContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +#define CONST(name, help, val, unit) { name, help, 0, AV_OPT_TYPE_CONST, {.i64=val}, INT_MIN, INT_MAX, FLAGS, unit } + +static const AVOption fastdeint_options[] = { + { "mode", "specify the deinterlacing mode", OFFSET(mode), AV_OPT_TYPE_INT, {.i64=MODE_BLEND}, 0, MODE_MAX-1, FLAGS, "mode" }, + CONST("discard", "discard bottom frame", MODE_DISCARD, "mode"), + CONST("mean", "half resolution blender", MODE_MEAN, "mode"), + CONST("blend", "full resolution blender", MODE_BLEND, "mode"), + CONST("bob", "bob doubler", MODE_BOB, "mode"), + CONST("linear", "bob doubler with linear interpolation", MODE_LINEAR, "mode"), + + { NULL } +}; + +AVFILTER_DEFINE_CLASS(fastdeint); + +static const AVFilterPad fastdeint_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad fastdeint_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_props, + .request_frame = request_frame + }, + { NULL } +}; + +AVFilter ff_vf_fastdeint = { + .name = "fastdeint", + .description = NULL_IF_CONFIG_SMALL("fast deinterlacing algorithms"), + .priv_size = sizeof(FastDeintContext), + .priv_class = &fastdeint_class, + .uninit = uninit, + .query_formats = query_formats, + .inputs = fastdeint_inputs, + .outputs = fastdeint_outputs, +}; -- 2.20.1

5 8

[PATCH 0/5] libdc1394 enhancements
by Forest Crossman 28 Oct '20

28 Oct '20

I was having trouble with libdc1394 the first time I used it (e.g., setting unsupported video modes/framerates on cameras, trying to use multiple cameras, tring to figure out how to set the arguments so the command would actually work, etc.) so I thought I'd go ahead and add some ease-of-use enhancements to to the module and some examples to the documentation to help others get started. I also wound up finding a bug with some early PTS wrapping, and I guess some encoders masked the issue because when I tried dumping the raw video to an MKV file the matroska encoder complained _very loudly_ about the non-monotonic PTS/DTS values. I had some trouble determining the best way to write the mode/framerate support checks and how to present the information to the user (errors are obvious, but I didn't know when would be a good time to list the supported modes and how best to print them), so if anyone has any suggestions on how to make that more "elegant" I'd be happy to hear them! I also had some trouble with the camera GUID:unit parsing--it works, but if the input is slightly incorrect the parsing will fail in strange ways. It will also successfully parse incorrect specifiers like 0xabcdef001122334455.0 (the period should be a colon) and 0x0xabcdef001122334455 (there's an extra "0x" prefix) and I'd really prefer it if it would error out and alert the user instead of fail-succeeding silently, but I'm not very well versed in C's string parsing so I'm not sure how I can improve that without significantly increasing the code complexity. Forest Crossman (5): libdc1394: Enable listing sources libdc1394: Verify the camera supports the selected mode and framerate libdc1394: Enable specifying a camera by GUID[:unit] doc/indevs: Add examples for libdc1394 libdc1394: Fix PTS wrapping doc/indevs.texi | 54 ++++++++++ libavdevice/libdc1394.c | 217 ++++++++++++++++++++++++++++++++++------ 2 files changed, 240 insertions(+), 31 deletions(-) -- 2.23.0

3 20