[PATCH] avfilter: add Audio Video Sync Test filter
Hi, patch attached.
Hi Paul, Can you please point me to some information how this filter works? Best, Alex. On Wed, Aug 16, 2017 at 12:08 PM, Paul B Mahol <onemda@gmail.com> wrote:
Hi,
patch attached.
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
On 8/17/17, Alex Giladi <alex.giladi@gmail.com> wrote:
Hi Paul, Can you please point me to some information how this filter works? Best, Alex.
How to run: mpv avdevice://lavfi:avsynctest[out1] Are you interested in something else?
Le nonidi 29 thermidor, an CCXXV, Paul B Mahol a écrit :
Hi,
patch attached.
From 8d3876499b93e61ec1aec39adfc3c3be8d622464 Mon Sep 17 00:00:00 2001 From: Paul B Mahol <onemda@gmail.com> Date: Fri, 28 Apr 2017 20:18:38 +0200 Subject: [PATCH] avfilter: add Audio Video Sync Test filter
Signed-off-by: Paul B Mahol <onemda@gmail.com> --- libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/src_avsynctest.c | 314 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 316 insertions(+) create mode 100644 libavfilter/src_avsynctest.c
Documentation missing.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 545b871cd7..aaf7375303 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -373,6 +373,7 @@ OBJS-$(CONFIG_SHOWWAVESPIC_FILTER) += avf_showwaves.o OBJS-$(CONFIG_SPECTRUMSYNTH_FILTER) += vaf_spectrumsynth.o window_func.o
# multimedia sources
+OBJS-$(CONFIG_AVSYNCTEST_FILTER) += src_avsynctest.o
It is a source, not a filter.
OBJS-$(CONFIG_AMOVIE_FILTER) += src_movie.o OBJS-$(CONFIG_MOVIE_FILTER) += src_movie.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index f1dacaffa8..32f4049b30 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -384,6 +384,7 @@ static void register_all(void) REGISTER_FILTER(SPECTRUMSYNTH, spectrumsynth, vaf);
/* multimedia sources */ + REGISTER_FILTER(AVSYNCTEST, avsynctest, avsrc); REGISTER_FILTER(AMOVIE, amovie, avsrc); REGISTER_FILTER(MOVIE, movie, avsrc);
diff --git a/libavfilter/src_avsynctest.c b/libavfilter/src_avsynctest.c new file mode 100644 index 0000000000..2f95976cbb --- /dev/null +++ b/libavfilter/src_avsynctest.c @@ -0,0 +1,314 @@ +/* + * Copyright (c) 2017 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/ffmath.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" +#include "libavutil/intreadwrite.h" +#include "libavutil/parseutils.h" +#include "libavutil/timestamp.h" +#include "libavutil/xga_font_data.h" +#include "avfilter.h" +#include "drawutils.h" +#include "formats.h" +#include "internal.h" +#include "audio.h" +#include "video.h" + +typedef struct AVSyncTestContext { + const AVClass *class; + + int w, h; + AVRational frame_rate; + int sample_rate; + int64_t apts, vpts, prev_vpts; + float amplitude; + int period; + int delay; + int cycle; + + int previ; + int beep; + int beep_duration; + int flash; + int dir; + float vdelay, delay_max, delay_min; + float delay_range; + + FFDrawContext draw; + FFDrawColor white; + FFDrawColor black; + FFDrawColor gray; +} AVSyncTestContext; + +#define OFFSET(x) offsetof(AVSyncTestContext, x) +#define A AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +#define V AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption avsynctest_options[] = { + {"size", "set frame size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str="hd720"}, 0, 0, V }, + {"s", "set frame size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str="hd720"}, 0, 0, V }, + {"framerate", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="30"}, 0,INT_MAX, V }, + {"fr", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="30"}, 0,INT_MAX, V }, + {"samplerate", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100},8000,192000, A }, + {"sr", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100},8000,192000, A }, + {"amplitude", "set amplitude", OFFSET(amplitude), AV_OPT_TYPE_FLOAT, {.dbl=.5}, 0., 1., A }, + {"a", "set amplitude", OFFSET(amplitude), AV_OPT_TYPE_FLOAT, {.dbl=.5}, 0., 1., A }, + {"period", "set beep period", OFFSET(period), AV_OPT_TYPE_INT, {.i64=3}, 1, 99., A }, + {"p", "set beep period", OFFSET(period), AV_OPT_TYPE_INT, {.i64=3}, 1, 99., A }, + {"delay", "set flash delay", OFFSET(delay), AV_OPT_TYPE_INT, {.i64=0}, -30, 30, V }, + {"d", "set flash delay", OFFSET(delay), AV_OPT_TYPE_INT, {.i64=0}, -30, 30, V }, + {"cycle", "set delay cycle", OFFSET(cycle), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, V }, + {"c", "set delay cycle", OFFSET(cycle), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, V }, + {NULL}, +}; + +AVFILTER_DEFINE_CLASS(avsynctest); + +static av_cold int query_formats(AVFilterContext *ctx) +{ + AVSyncTestContext *s = ctx->priv; + static const int64_t chlayouts[] = { AV_CH_LAYOUT_MONO, -1 }; + int sample_rates[] = { s->sample_rate, -1 }; + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_FLT, + AV_SAMPLE_FMT_NONE + }; + AVFilterFormats *formats; + AVFilterChannelLayouts *layouts; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + if ((ret = ff_formats_ref(formats, &ctx->outputs[0]->in_formats)) < 0) + return ret; + + formats = ff_draw_supported_pixel_formats(0); + if (!formats) + return AVERROR(ENOMEM); + if ((ret = ff_formats_ref(formats, &ctx->outputs[1]->in_formats)) < 0) + return ret; + + layouts = avfilter_make_format64_list(chlayouts); + if (!layouts) + return AVERROR(ENOMEM); + ret = ff_set_common_channel_layouts(ctx, layouts); + if (ret < 0) + return ret; + + formats = ff_make_format_list(sample_rates); + if (!formats) + return AVERROR(ENOMEM); + return ff_set_common_samplerates(ctx, formats); +} + +static av_cold int aconfig_props(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + + outlink->sample_rate = s->sample_rate; + outlink->time_base = (AVRational){1, s->sample_rate}; + + s->beep_duration = s->sample_rate * s->frame_rate.den / s->frame_rate.num ; + + return 0; +} + +static av_cold int config_props(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + + outlink->w = s->w; + outlink->h = s->h; + outlink->time_base = av_inv_q(s->frame_rate); + outlink->sample_aspect_ratio = (AVRational) {1, 1}; + s->delay_min = -av_q2d(s->frame_rate) / 2; + s->delay_max = -s->delay_min; + s->delay_range = s->delay_max - s->delay_min; + s->vdelay = s->delay; + s->dir = 1; + + ff_draw_init(&s->draw, outlink->format, 0); + + ff_draw_color(&s->draw, &s->black, (uint8_t[]){ 0, 0, 0, 255} ); + ff_draw_color(&s->draw, &s->white, (uint8_t[]){ 255, 255, 255, 255} ); + ff_draw_color(&s->draw, &s->gray, (uint8_t[]){ 128, 128, 128, 255} ); + + return 0; +} + +static int arequest_frame(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + AVFrame *out = ff_get_audio_buffer(outlink, 1024); + const float a = s->amplitude; + float *dst; + int i; + if (!out) + return AVERROR(ENOMEM); + + out->pts = s->apts; + dst = (float *)out->data[0]; + + for (i = 0; i < 1024; i++) {
+ if (((s->apts + i) % (s->period * s->sample_rate)) == 0) + s->beep = 1; + if (s->beep) { + dst[i] = a * sinf(800 * 2 * M_PI * av_q2d(outlink->time_base) * (s->apts + i)); + s->beep++; + } else { + dst[i] = 0; + } + if (s->beep >= s->beep_duration) { + s->beep = 0; + }
The use of floats makes this filter unusable for any kind of automated testing, except with inconvenient approximate comparisons. The sine audio source gives a similar output but does not have this problem.
+ } + s->apts += out->nb_samples; + + return ff_filter_frame(outlink, out); +} + +static void draw_text(FFDrawContext *draw, AVFrame *out, FFDrawColor *color, + int x0, int y0, const uint8_t *text) +{ + int x = x0; + + for (; *text; text++) { + if (*text == '\n') { + x = x0; + y0 += 8; + continue; + } + ff_blend_mask(draw, color, out->data, out->linesize, + out->width, out->height, + avpriv_cga_font + *text * 8, 1, 8, 8, 0, 0, x, y0); + x += 8; + } +} + +static int request_frame(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + AVFrame *out = ff_get_video_buffer(outlink, s->w, s->h); + int step = outlink->w / s->delay_range; + double intpart; + char text[128]; + int width, offset, i; + + if (!out) + return AVERROR(ENOMEM); + + ff_fill_rectangle(&s->draw, &s->black, out->data, out->linesize, + 0, 0, outlink->w, outlink->h); + + snprintf(text, sizeof(text), "FRN: %"PRId64"", s->vpts); + draw_text(&s->draw, out, &s->white, outlink->w * .1, outlink->h * .1, text); + + snprintf(text, sizeof(text), "SEC: %s", av_ts2timestr(s->vpts, &outlink->time_base)); + draw_text(&s->draw, out, &s->white, outlink->w * .1, outlink->h * .9, text); + + snprintf(text, sizeof(text), "DLY: %d", (int)s->vdelay); + draw_text(&s->draw, out, &s->white, outlink->w * .9 - strlen(text) * 8, outlink->h * .9, text); + + snprintf(text, sizeof(text), "FPS: %d/%d", s->frame_rate.num, s->frame_rate.den); + draw_text(&s->draw, out, &s->white, outlink->w * .9 - strlen(text) * 8, outlink->h * .1, text); + + snprintf(text, sizeof(text), "P: %d", s->period); + draw_text(&s->draw, out, &s->gray, outlink->w * .5 - strlen(text) * 4, outlink->h * .9, text); + + snprintf(text, sizeof(text), "SR: %d", s->sample_rate); + draw_text(&s->draw, out, &s->gray, outlink->w * .5 - strlen(text) * 4, outlink->h * .1, text); + + snprintf(text, sizeof(text), "A: %f", s->amplitude); + draw_text(&s->draw, out, &s->gray, outlink->w * .1, outlink->h * .5, text); + + snprintf(text, sizeof(text), "WxH: %dx%d", outlink->w, outlink->h); + draw_text(&s->draw, out, &s->gray, outlink->w * .9 - strlen(text) * 8, outlink->h * .5, text); + + modf(av_q2d(outlink->time_base) * (s->vpts + s->vdelay), &intpart); + + width = out->width - (out->width * .026); + offset = (out->width - width) / 2; + ff_fill_rectangle(&s->draw, &s->white, out->data, out->linesize, + offset + step * (((int)(s->delay_max + s->vpts) * s->frame_rate.den) % s->frame_rate.num) / s->frame_rate.den, + outlink->h * .705, outlink->w * .025, outlink->h * .04); + + if (s->previ + 1 == intpart) { + s->flash++; + if (s->flash >= s->period) { + if (s->cycle) { + s->vdelay += s->dir; + } + if (s->vdelay >= s->delay_max) { + s->dir = -1; + s->prev_vpts = s->vpts; + } else if (s->vdelay <= s->delay_min) { + s->dir = 1; + s->prev_vpts = s->vpts; + } + ff_fill_rectangle(&s->draw, &s->white, out->data, out->linesize, + outlink->w * .35, outlink->h * .35, outlink->w * .25, outlink->h * .25); + s->flash = 0; + } + } + s->previ = intpart; + + for (i = 0; i < s->delay_range; i++) { + ff_fill_rectangle(&s->draw, &s->white, out->data, out->linesize, + offset + step * i, outlink->h * .7, 1, outlink->h * .05); + } + + out->pts = s->vpts++; + + return ff_filter_frame(outlink, out); +} + +static const AVFilterPad avsynctest_outputs[] = { + { + .name = "audio", + .type = AVMEDIA_TYPE_AUDIO, + .request_frame = arequest_frame, + .config_props = aconfig_props, + }, + { + .name = "video", + .type = AVMEDIA_TYPE_VIDEO, + .request_frame = request_frame, + .config_props = config_props, + }, + { NULL } +}; + +AVFilter ff_avsrc_avsynctest = { + .name = "avsynctest", + .description = NULL_IF_CONFIG_SMALL("Generate an Audio Video Sync Test."), + .query_formats = query_formats, + .priv_size = sizeof(AVSyncTestContext), + .inputs = NULL, + .outputs = avsynctest_outputs, + .priv_class = &avsynctest_class, +};
Can you explain what this source allows that testsrc2+sine does not? Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le nonidi 29 thermidor, an CCXXV, Paul B Mahol a écrit :
Hi,
patch attached.
From 8d3876499b93e61ec1aec39adfc3c3be8d622464 Mon Sep 17 00:00:00 2001 From: Paul B Mahol <onemda@gmail.com> Date: Fri, 28 Apr 2017 20:18:38 +0200 Subject: [PATCH] avfilter: add Audio Video Sync Test filter
Signed-off-by: Paul B Mahol <onemda@gmail.com> --- libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/src_avsynctest.c | 314 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 316 insertions(+) create mode 100644 libavfilter/src_avsynctest.c
Documentation missing.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 545b871cd7..aaf7375303 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -373,6 +373,7 @@ OBJS-$(CONFIG_SHOWWAVESPIC_FILTER) += avf_showwaves.o OBJS-$(CONFIG_SPECTRUMSYNTH_FILTER) += vaf_spectrumsynth.o window_func.o
# multimedia sources
+OBJS-$(CONFIG_AVSYNCTEST_FILTER) += src_avsynctest.o
It is a source, not a filter.
Try harder to explain what you think.
OBJS-$(CONFIG_AMOVIE_FILTER) += src_movie.o OBJS-$(CONFIG_MOVIE_FILTER) += src_movie.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index f1dacaffa8..32f4049b30 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -384,6 +384,7 @@ static void register_all(void) REGISTER_FILTER(SPECTRUMSYNTH, spectrumsynth, vaf);
/* multimedia sources */ + REGISTER_FILTER(AVSYNCTEST, avsynctest, avsrc); REGISTER_FILTER(AMOVIE, amovie, avsrc); REGISTER_FILTER(MOVIE, movie, avsrc);
diff --git a/libavfilter/src_avsynctest.c b/libavfilter/src_avsynctest.c new file mode 100644 index 0000000000..2f95976cbb --- /dev/null +++ b/libavfilter/src_avsynctest.c @@ -0,0 +1,314 @@ +/* + * Copyright (c) 2017 Paul B Mahol + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/ffmath.h" +#include "libavutil/opt.h" +#include "libavutil/imgutils.h" +#include "libavutil/intreadwrite.h" +#include "libavutil/parseutils.h" +#include "libavutil/timestamp.h" +#include "libavutil/xga_font_data.h" +#include "avfilter.h" +#include "drawutils.h" +#include "formats.h" +#include "internal.h" +#include "audio.h" +#include "video.h" + +typedef struct AVSyncTestContext { + const AVClass *class; + + int w, h; + AVRational frame_rate; + int sample_rate; + int64_t apts, vpts, prev_vpts; + float amplitude; + int period; + int delay; + int cycle; + + int previ; + int beep; + int beep_duration; + int flash; + int dir; + float vdelay, delay_max, delay_min; + float delay_range; + + FFDrawContext draw; + FFDrawColor white; + FFDrawColor black; + FFDrawColor gray; +} AVSyncTestContext; + +#define OFFSET(x) offsetof(AVSyncTestContext, x) +#define A AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +#define V AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM + +static const AVOption avsynctest_options[] = { + {"size", "set frame size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str="hd720"}, 0, 0, V }, + {"s", "set frame size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str="hd720"}, 0, 0, V }, + {"framerate", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="30"}, 0,INT_MAX, V }, + {"fr", "set frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, {.str="30"}, 0,INT_MAX, V }, + {"samplerate", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100},8000,192000, A }, + {"sr", "set sample rate", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64=44100},8000,192000, A }, + {"amplitude", "set amplitude", OFFSET(amplitude), AV_OPT_TYPE_FLOAT, {.dbl=.5}, 0., 1., A }, + {"a", "set amplitude", OFFSET(amplitude), AV_OPT_TYPE_FLOAT, {.dbl=.5}, 0., 1., A }, + {"period", "set beep period", OFFSET(period), AV_OPT_TYPE_INT, {.i64=3}, 1, 99., A }, + {"p", "set beep period", OFFSET(period), AV_OPT_TYPE_INT, {.i64=3}, 1, 99., A }, + {"delay", "set flash delay", OFFSET(delay), AV_OPT_TYPE_INT, {.i64=0}, -30, 30, V }, + {"d", "set flash delay", OFFSET(delay), AV_OPT_TYPE_INT, {.i64=0}, -30, 30, V }, + {"cycle", "set delay cycle", OFFSET(cycle), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, V }, + {"c", "set delay cycle", OFFSET(cycle), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, V }, + {NULL}, +}; + +AVFILTER_DEFINE_CLASS(avsynctest); + +static av_cold int query_formats(AVFilterContext *ctx) +{ + AVSyncTestContext *s = ctx->priv; + static const int64_t chlayouts[] = { AV_CH_LAYOUT_MONO, -1 }; + int sample_rates[] = { s->sample_rate, -1 }; + static const enum AVSampleFormat sample_fmts[] = { + AV_SAMPLE_FMT_FLT, + AV_SAMPLE_FMT_NONE + }; + AVFilterFormats *formats; + AVFilterChannelLayouts *layouts; + int ret; + + formats = ff_make_format_list(sample_fmts); + if (!formats) + return AVERROR(ENOMEM); + if ((ret = ff_formats_ref(formats, &ctx->outputs[0]->in_formats)) < 0) + return ret; + + formats = ff_draw_supported_pixel_formats(0); + if (!formats) + return AVERROR(ENOMEM); + if ((ret = ff_formats_ref(formats, &ctx->outputs[1]->in_formats)) < 0) + return ret; + + layouts = avfilter_make_format64_list(chlayouts); + if (!layouts) + return AVERROR(ENOMEM); + ret = ff_set_common_channel_layouts(ctx, layouts); + if (ret < 0) + return ret; + + formats = ff_make_format_list(sample_rates); + if (!formats) + return AVERROR(ENOMEM); + return ff_set_common_samplerates(ctx, formats); +} + +static av_cold int aconfig_props(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + + outlink->sample_rate = s->sample_rate; + outlink->time_base = (AVRational){1, s->sample_rate}; + + s->beep_duration = s->sample_rate * s->frame_rate.den / s->frame_rate.num ; + + return 0; +} + +static av_cold int config_props(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + + outlink->w = s->w; + outlink->h = s->h; + outlink->time_base = av_inv_q(s->frame_rate); + outlink->sample_aspect_ratio = (AVRational) {1, 1}; + s->delay_min = -av_q2d(s->frame_rate) / 2; + s->delay_max = -s->delay_min; + s->delay_range = s->delay_max - s->delay_min; + s->vdelay = s->delay; + s->dir = 1; + + ff_draw_init(&s->draw, outlink->format, 0); + + ff_draw_color(&s->draw, &s->black, (uint8_t[]){ 0, 0, 0, 255} ); + ff_draw_color(&s->draw, &s->white, (uint8_t[]){ 255, 255, 255, 255} ); + ff_draw_color(&s->draw, &s->gray, (uint8_t[]){ 128, 128, 128, 255} ); + + return 0; +} + +static int arequest_frame(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + AVFrame *out = ff_get_audio_buffer(outlink, 1024); + const float a = s->amplitude; + float *dst; + int i; + if (!out) + return AVERROR(ENOMEM); + + out->pts = s->apts; + dst = (float *)out->data[0]; + + for (i = 0; i < 1024; i++) {
+ if (((s->apts + i) % (s->period * s->sample_rate)) == 0) + s->beep = 1; + if (s->beep) { + dst[i] = a * sinf(800 * 2 * M_PI * av_q2d(outlink->time_base) * (s->apts + i)); + s->beep++; + } else { + dst[i] = 0; + } + if (s->beep >= s->beep_duration) { + s->beep = 0; + }
The use of floats makes this filter unusable for any kind of automated testing, except with inconvenient approximate comparisons.
The sine audio source gives a similar output but does not have this problem.
+ } + s->apts += out->nb_samples; + + return ff_filter_frame(outlink, out); +} + +static void draw_text(FFDrawContext *draw, AVFrame *out, FFDrawColor *color, + int x0, int y0, const uint8_t *text) +{ + int x = x0; + + for (; *text; text++) { + if (*text == '\n') { + x = x0; + y0 += 8; + continue; + } + ff_blend_mask(draw, color, out->data, out->linesize, + out->width, out->height, + avpriv_cga_font + *text * 8, 1, 8, 8, 0, 0, x, y0); + x += 8; + } +} + +static int request_frame(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVSyncTestContext *s = ctx->priv; + AVFrame *out = ff_get_video_buffer(outlink, s->w, s->h); + int step = outlink->w / s->delay_range; + double intpart; + char text[128]; + int width, offset, i; + + if (!out) + return AVERROR(ENOMEM); + + ff_fill_rectangle(&s->draw, &s->black, out->data, out->linesize, + 0, 0, outlink->w, outlink->h); + + snprintf(text, sizeof(text), "FRN: %"PRId64"", s->vpts); + draw_text(&s->draw, out, &s->white, outlink->w * .1, outlink->h * .1, text); + + snprintf(text, sizeof(text), "SEC: %s", av_ts2timestr(s->vpts, &outlink->time_base)); + draw_text(&s->draw, out, &s->white, outlink->w * .1, outlink->h * .9, text); + + snprintf(text, sizeof(text), "DLY: %d", (int)s->vdelay); + draw_text(&s->draw, out, &s->white, outlink->w * .9 - strlen(text) * 8, outlink->h * .9, text); + + snprintf(text, sizeof(text), "FPS: %d/%d", s->frame_rate.num, s->frame_rate.den); + draw_text(&s->draw, out, &s->white, outlink->w * .9 - strlen(text) * 8, outlink->h * .1, text); + + snprintf(text, sizeof(text), "P: %d", s->period); + draw_text(&s->draw, out, &s->gray, outlink->w * .5 - strlen(text) * 4, outlink->h * .9, text); + + snprintf(text, sizeof(text), "SR: %d", s->sample_rate); + draw_text(&s->draw, out, &s->gray, outlink->w * .5 - strlen(text) * 4, outlink->h * .1, text); + + snprintf(text, sizeof(text), "A: %f", s->amplitude); + draw_text(&s->draw, out, &s->gray, outlink->w * .1, outlink->h * .5, text); + + snprintf(text, sizeof(text), "WxH: %dx%d", outlink->w, outlink->h); + draw_text(&s->draw, out, &s->gray, outlink->w * .9 - strlen(text) * 8, outlink->h * .5, text); + + modf(av_q2d(outlink->time_base) * (s->vpts + s->vdelay), &intpart); + + width = out->width - (out->width * .026); + offset = (out->width - width) / 2; + ff_fill_rectangle(&s->draw, &s->white, out->data, out->linesize, + offset + step * (((int)(s->delay_max + s->vpts) * s->frame_rate.den) % s->frame_rate.num) / s->frame_rate.den, + outlink->h * .705, outlink->w * .025, outlink->h * .04); + + if (s->previ + 1 == intpart) { + s->flash++; + if (s->flash >= s->period) { + if (s->cycle) { + s->vdelay += s->dir; + } + if (s->vdelay >= s->delay_max) { + s->dir = -1; + s->prev_vpts = s->vpts; + } else if (s->vdelay <= s->delay_min) { + s->dir = 1; + s->prev_vpts = s->vpts; + } + ff_fill_rectangle(&s->draw, &s->white, out->data, out->linesize, + outlink->w * .35, outlink->h * .35, outlink->w * .25, outlink->h * .25); + s->flash = 0; + } + } + s->previ = intpart; + + for (i = 0; i < s->delay_range; i++) { + ff_fill_rectangle(&s->draw, &s->white, out->data, out->linesize, + offset + step * i, outlink->h * .7, 1, outlink->h * .05); + } + + out->pts = s->vpts++; + + return ff_filter_frame(outlink, out); +} + +static const AVFilterPad avsynctest_outputs[] = { + { + .name = "audio", + .type = AVMEDIA_TYPE_AUDIO, + .request_frame = arequest_frame, + .config_props = aconfig_props, + }, + { + .name = "video", + .type = AVMEDIA_TYPE_VIDEO, + .request_frame = request_frame, + .config_props = config_props, + }, + { NULL } +}; + +AVFilter ff_avsrc_avsynctest = { + .name = "avsynctest", + .description = NULL_IF_CONFIG_SMALL("Generate an Audio Video Sync Test."), + .query_formats = query_formats, + .priv_size = sizeof(AVSyncTestContext), + .inputs = NULL, + .outputs = avsynctest_outputs, + .priv_class = &avsynctest_class, +};
Can you explain what this source allows that testsrc2+sine does not?
If you actually tried you would know.
Regards,
-- Nicolas George
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
Try harder to explain what you think.
This "filter" has no input. All similar "filters" are called sources and implemented in files named accordingly.
If you actually tried you would know.
I am sorry, but I do not accept that as an answer. Please explain why all this extra code is useful compared to what already exists. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
Try harder to explain what you think.
This "filter" has no input. All similar "filters" are called sources and implemented in files named accordingly.
So it is. So unless you tell what exactly you mean I can not help you.
If you actually tried you would know.
I am sorry, but I do not accept that as an answer. Please explain why all this extra code is useful compared to what already exists.
testsrc(2) are useless for lipsync testing, when one needs to know exact time something is played in wrong time.
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
So it is. So unless you tell what exactly you mean I can not help you.
My bad.
testsrc(2) are useless for lipsync testing, when one needs to know exact time something is played in wrong time.
Yes, testsrc alone is not enough. But you did not read carefully enough: I mentioned testsrc2+sine. The sine source can optionally generate a periodic beep, just like your proposal. I grant you, the duration and period of the beep cannot be configured, but that would be easy to add. And sine is bit-exact and faster. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
So it is. So unless you tell what exactly you mean I can not help you.
My bad.
testsrc(2) are useless for lipsync testing, when one needs to know exact time something is played in wrong time.
Yes, testsrc alone is not enough. But you did not read carefully enough: I mentioned testsrc2+sine. The sine source can optionally generate a periodic beep, just like your proposal. I grant you, the duration and period of the beep cannot be configured, but that would be easy to add. And sine is bit-exact and faster.
And how would you show delay introduced with video source filter? testsrc(2) does not show anything resembling that. Look at those youtube videos: https://www.youtube.com/results?search_query=video+sync
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
And how would you show delay introduced with video source filter? testsrc(2) does not show anything resembling that.
I do not understand what you mean. The output of "ffmpeg -lavfi 'testsrc2;sine'" looks functionally identical to the video you point. Please be more specific, with more than two short lines of answer. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
And how would you show delay introduced with video source filter? testsrc(2) does not show anything resembling that.
I do not understand what you mean. The output of "ffmpeg -lavfi 'testsrc2;sine'" looks functionally identical to the video you point. Please be more specific, with more than two short lines of answer.
How do you know when beep and blink should happen in your example? What is blink for testsrc2? Where in testsrc2 you can see delay in numeric form?
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
How do you know when beep and blink should happen in your example?
What is blink for testsrc2? Where in testsrc2 you can see delay in numeric form?
Have you tried? You have a numeric timestamp and frame counter in the upper left corner and a beating cross, both in sync with the beeps. Please tell me what you are missing here. With more than two lines please. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
How do you know when beep and blink should happen in your example?
What is blink for testsrc2? Where in testsrc2 you can see delay in numeric form?
Have you tried? You have a numeric timestamp and frame counter in the upper left corner and a beating cross, both in sync with the beeps. Please tell me what you are missing here. With more than two lines please.
Exact delay in number of frames and that converted to milliseconds. Also one can not set custom delay between beep and blink. Also beating cross is far from being easily noticed like blink.
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
Exact delay in number of frames and that converted to milliseconds.
Already present. I asked you if you tested, obviously you did not.
Also one can not set custom delay between beep and blink.
I already mentioned that. You can propose a patch to sine to add that option.
Also beating cross is far from being easily noticed like blink.
I do not agree. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
Exact delay in number of frames and that converted to milliseconds.
Already present. I asked you if you tested, obviously you did not.
Sorry but where is that present? I nowhere can determine exact delay in number of frames versus sine beep.
Also one can not set custom delay between beep and blink.
I already mentioned that. You can propose a patch to sine to add that option.
It is not sine but testsrc2 one that would need modification.
Also beating cross is far from being easily noticed like blink.
I do not agree.
Why you do not agree? When is actual beep when beating cross appears?
Regards,
-- Nicolas George
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
Sorry but where is that present? I nowhere can determine exact delay in number of frames versus sine beep.
In the upper-left corner, you have a microsecond timestamp and a frame counter.
It is not sine but testsrc2 one that would need modification.
Care to explain? The beep is produced by sine. Why would you want to modify testsrc2?
Why you do not agree? When is actual beep when beating cross appears?
The beep is when the cross is fully contracted. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
Sorry but where is that present? I nowhere can determine exact delay in number of frames versus sine beep.
In the upper-left corner, you have a microsecond timestamp and a frame counter.
That is not delay between blink and beep. But video timestamps.
It is not sine but testsrc2 one that would need modification.
Care to explain? The beep is produced by sine. Why would you want to modify testsrc2?
For lipsync one would need to alter so blinks happens at exact same time as beeps. Say delay is 10 frames, you modify filter to produce delayed blinks.
Why you do not agree? When is actual beep when beating cross appears?
The beep is when the cross is fully contracted.
That is far for being user friendly and obvious.
Regards,
-- Nicolas George _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
That is not delay between blink and beep. But video timestamps.
Since the beeps happen every second, the delay is the fractional part, or the remainder of the frame count divided by the frame rate. But if you think this arithmetic is too complex for our users, feel free to propose an option to add it on testsrc2.
For lipsync one would need to alter so blinks happens at exact same time as beeps.
They do.
Say delay is 10 frames, you modify filter to produce delayed blinks.
I think it should be a job for setpts, but if you consider that too complex, again, I am not against a time delay option to testsrc2.
The beep is when the cross is fully contracted. That is far for being user friendly and obvious.
It looks as obvious as the bumping dot on the videos you showed, in my opinion. Anyway, I am not against you wasting your time adding yet another test pattern, but not as proposed in this patch: use the framework of all the test patterns and make it bit-exact. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
That is not delay between blink and beep. But video timestamps.
Since the beeps happen every second, the delay is the fractional part, or the remainder of the frame count divided by the frame rate. But if you think this arithmetic is too complex for our users, feel free to propose an option to add it on testsrc2.
For lipsync one would need to alter so blinks happens at exact same time as beeps.
They do.
Say delay is 10 frames, you modify filter to produce delayed blinks.
I think it should be a job for setpts, but if you consider that too complex, again, I am not against a time delay option to testsrc2.
The beep is when the cross is fully contracted. That is far for being user friendly and obvious.
It looks as obvious as the bumping dot on the videos you showed, in my opinion.
Anyway, I am not against you wasting your time adding yet another test pattern, but not as proposed in this patch: use the framework of all the test patterns and make it bit-exact.
I like idea of having single A/V filter to do this job, not combinations where users might become confused. Also this filter displays audio parameters in video output, which would become pain to do in combination of filters.
Regards,
-- Nicolas George _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
I like idea of having single A/V filter to do this job, not combinations where users might become confused.
I do not agree. Adding yet another filter causes confusion in itself. Not counting the extra boilerplate code (I suggest you permanently disable the copy-paste feature of your development editor ;-). Documenting the combination would have the same result. Furthermore, having this in a single filter would possibly cause the misapprehension that something special happens there, while in truth it is no different from two individual filters. Also, two individual filters would be able to run in separate threads when filter-level threading will be implemented. Regards, -- Nicolas George
On 8/19/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
I like idea of having single A/V filter to do this job, not combinations where users might become confused.
I do not agree. Adding yet another filter causes confusion in itself. Not counting the extra boilerplate code (I suggest you permanently disable the copy-paste feature of your development editor ;-). Documenting the combination would have the same result.
You completely ignored and removed other aspects of my reply.
Furthermore, having this in a single filter would possibly cause the misapprehension that something special happens there, while in truth it is no different from two individual filters.
Something special, like one can control both audio and video aspect at once and provide same info to output.
Also, two individual filters would be able to run in separate threads when filter-level threading will be implemented.
*when* Also that matter less.
Regards,
-- Nicolas George _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
You completely ignored and removed other aspects of my reply.
You still are being vague.
Something special, like one can control both audio and video aspect at once and provide same info to output.
On the usability / convenience question, we obviously disagree. I consider that multiplying the test source filters is a more serious source of annoyance and confusion for the users, but it is a matter of appreciation and you obviously disagree. For that kind of question the normal policy would be to ask for the appreciation of other people, especially the ones who deal with users on a regular basis, before committing. As for the technical issues, there are two, and the answer must be technical: This filter includes some amount of boilerplate code, which is a maintenance burden. Please make more efforts to reduce that. In this particular instance, I believe using the framework that Stefano designed for test sources would reduce the boilerplate code. This filter is not bit-exact. A test source needs very strongly to be bit-exact. There is in the code base all that is needed to make it so, and that would make it faster as well. Regards, -- Nicolas George
On 8/23/17, Nicolas George <george@nsup.org> wrote:
Le duodi 2 fructidor, an CCXXV, Paul B Mahol a écrit :
You completely ignored and removed other aspects of my reply.
You still are being vague.
Something special, like one can control both audio and video aspect at once and provide same info to output.
On the usability / convenience question, we obviously disagree. I consider that multiplying the test source filters is a more serious source of annoyance and confusion for the users, but it is a matter of appreciation and you obviously disagree. For that kind of question the normal policy would be to ask for the appreciation of other people, especially the ones who deal with users on a regular basis, before committing.
I was asked multiple times to implement this filter.
As for the technical issues, there are two, and the answer must be technical:
This filter includes some amount of boilerplate code, which is a maintenance burden. Please make more efforts to reduce that. In this particular instance, I believe using the framework that Stefano designed for test sources would reduce the boilerplate code.
What framework? There is only framework for video source filter which outputs static frames.
This filter is not bit-exact. A test source needs very strongly to be bit-exact. There is in the code base all that is needed to make it so, and that would make it faster as well.
I could make use of sine filter code.
Le sextidi 6 fructidor, an CCXXV, Paul B Mahol a écrit :
I was asked multiple times to implement this filter.
I was not referring to the advice of single users with a tunnel vision of their own use case.
What framework? There is only framework for video source filter which outputs static frames.
You are mistaken: testsrc and testsrc2 use that framework and output variable frames.
I could make use of sine filter code.
Exactly. Regards, -- Nicolas George
On 8/23/17, Nicolas George <george@nsup.org> wrote:
Le sextidi 6 fructidor, an CCXXV, Paul B Mahol a écrit :
I was asked multiple times to implement this filter.
I was not referring to the advice of single users with a tunnel vision of their own use case.
What framework? There is only framework for video source filter which outputs static frames.
You are mistaken: testsrc and testsrc2 use that framework and output variable frames.
I could make use of sine filter code.
Exactly.
Regards,
I changed my mind, I will not follow Nicolas advice at all. Will apply this filter ASAP. I kindly ask Nicolas to stay away from my code for ever.
Paul B Mahol (12020-03-12):
I changed my mind, I will not follow Nicolas advice at all. Will apply this filter ASAP.
I kindly ask Nicolas to stay away from my code for ever.
This is not how it works. Re-send the patch for consideration by the whole project. -- Nicolas George
participants (3)
-
Alex Giladi -
Nicolas George -
Paul B Mahol