March 2016 - ffmpeg-devel - lists.ffmpeg.org

[PATCH] lavf/matroskadec: Demux the PixelCrop* values
by Nic Wolfe 30 Mar '20

30 Mar '20

The Matroska spec defines PixelCropTop, PixelCropBottom, PixelCropLeft, and PixelCropRight elements: https://www.matroska.org/technical/specs/index.html This commit adds support for demuxing these values so that applications using libav* are able to use them when playing the stream. They're added to the AVStream's metadata if they are set to something non-zero. My official patch is base64 encoded and attached but I will also include the diff below for (hopefully) convenience. Thanks, Nic diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c index d788232..72537df 100644 --- a/libavformat/matroskadec.c +++ b/libavformat/matroskadec.c @@ -139,6 +139,10 @@ typedef struct MatroskaTrackVideo { EbmlBin color_space; uint64_t stereo_mode; uint64_t alpha_mode; + uint64_t crop_bottom; + uint64_t crop_top; + uint64_t crop_left; + uint64_t crop_right; } MatroskaTrackVideo; typedef struct MatroskaTrackAudio { @@ -364,10 +368,10 @@ static const EbmlSyntax matroska_track_video[] = { { MATROSKA_ID_VIDEOPIXELHEIGHT, EBML_UINT, 0, offsetof(MatroskaTrackVideo, pixel_height) }, { MATROSKA_ID_VIDEOCOLORSPACE, EBML_BIN, 0, offsetof(MatroskaTrackVideo, color_space) }, { MATROSKA_ID_VIDEOALPHAMODE, EBML_UINT, 0, offsetof(MatroskaTrackVideo, alpha_mode) }, - { MATROSKA_ID_VIDEOPIXELCROPB, EBML_NONE }, - { MATROSKA_ID_VIDEOPIXELCROPT, EBML_NONE }, - { MATROSKA_ID_VIDEOPIXELCROPL, EBML_NONE }, - { MATROSKA_ID_VIDEOPIXELCROPR, EBML_NONE }, + { MATROSKA_ID_VIDEOPIXELCROPB, EBML_UINT, 0, offsetof(MatroskaTrackVideo, crop_bottom) }, + { MATROSKA_ID_VIDEOPIXELCROPT, EBML_UINT, 0, offsetof(MatroskaTrackVideo, crop_top) }, + { MATROSKA_ID_VIDEOPIXELCROPL, EBML_UINT, 0, offsetof(MatroskaTrackVideo, crop_left) }, + { MATROSKA_ID_VIDEOPIXELCROPR, EBML_UINT, 0, offsetof(MatroskaTrackVideo, crop_right) }, { MATROSKA_ID_VIDEODISPLAYUNIT, EBML_NONE }, { MATROSKA_ID_VIDEOFLAGINTERLACED, EBML_NONE }, { MATROSKA_ID_VIDEOSTEREOMODE, EBML_UINT, 0, offsetof(MatroskaTrackVideo, stereo_mode), { .u = MATROSKA_VIDEO_STEREOMODE_TYPE_NB } }, @@ -2152,6 +2156,16 @@ static int matroska_parse_tracks(AVFormatContext *s) if (track->video.stereo_mode && track->video.stereo_mode < MATROSKA_VIDEO_STEREOMODE_TYPE_NB) av_dict_set(&st->metadata, "stereo_mode", ff_matroska_video_stereo_mode[track->video.stereo_mode], 0); + /* export the matroska crop settings as metadata */ + if (track->video.crop_bottom != 0) + av_dict_set_int(&st->metadata, "crop_bottom", track->video.crop_bottom, 0); + if (track->video.crop_top != 0) + av_dict_set_int(&st->metadata, "crop_top", track->video.crop_top, 0); + if (track->video.crop_left != 0) + av_dict_set_int(&st->metadata, "crop_left", track->video.crop_left, 0); + if (track->video.crop_right != 0) + av_dict_set_int(&st->metadata, "crop_right", track->video.crop_right, 0); + /* export alpha mode flag as metadata tag */ if (track->video.alpha_mode) av_dict_set(&st->metadata, "alpha_mode", "1", 0);

5 7

[PATCH] lavfi: add realtime filter.
by Nicolas George 25 Oct '18

25 Oct '18

Similar to the -re option in ffmpeg that only works for input files. Can be used at any place in the filter graph. Signed-off-by: Nicolas George <george(a)nsup.org> --- Changelog | 1 + doc/filters.texi | 16 ++++++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/f_realtime.c | 130 +++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 151 insertions(+) create mode 100644 libavfilter/f_realtime.c diff --git a/Changelog b/Changelog index abaac19..1c8fe37 100644 --- a/Changelog +++ b/Changelog @@ -25,6 +25,7 @@ version <next>: - wve demuxer - zero-copy Intel QSV transcoding in ffmpeg - shuffleframes filter +- realtime filter version 2.8: diff --git a/doc/filters.texi b/doc/filters.texi index 5a35bde..141394c 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -13052,6 +13052,22 @@ following one, the permission might not be received as expected in that following filter. Inserting a @ref{format} or @ref{aformat} filter before the perms/aperms filter can avoid this problem. +@section realtime, arealtime + +Slow down filtering to match real time approximatively. + +These filters will pause the filtering for a variable amount of time to +match the output rate with the input timestamps. +They are similar to the @option{re} option to @code{ffmpeg}. + +They accept the following options: + +@table @option +@item limit +Time limit for the pauses. Any pause longer than that will be considered +a timestamp discontinuity and reset the timer. Default is 2 seconds. +@end table + @section select, aselect Select frames to pass in output. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index dd4f547..9c07d98 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -39,6 +39,7 @@ OBJS-$(CONFIG_ANULL_FILTER) += af_anull.o OBJS-$(CONFIG_APAD_FILTER) += af_apad.o OBJS-$(CONFIG_APERMS_FILTER) += f_perms.o OBJS-$(CONFIG_APHASER_FILTER) += af_aphaser.o generate_wave_table.o +OBJS-$(CONFIG_AREALTIME_FILTER) += f_realtime.o OBJS-$(CONFIG_ARESAMPLE_FILTER) += af_aresample.o OBJS-$(CONFIG_AREVERSE_FILTER) += f_reverse.o OBJS-$(CONFIG_ASELECT_FILTER) += f_select.o @@ -196,6 +197,7 @@ OBJS-$(CONFIG_PSNR_FILTER) += vf_psnr.o dualinput.o framesync. OBJS-$(CONFIG_PULLUP_FILTER) += vf_pullup.o OBJS-$(CONFIG_QP_FILTER) += vf_qp.o OBJS-$(CONFIG_RANDOM_FILTER) += vf_random.o +OBJS-$(CONFIG_REALTIME_FILTER) += f_realtime.o OBJS-$(CONFIG_REMOVEGRAIN_FILTER) += vf_removegrain.o OBJS-$(CONFIG_REMOVELOGO_FILTER) += bbox.o lswsutils.o lavfutils.o vf_removelogo.o OBJS-$(CONFIG_REPEATFIELDS_FILTER) += vf_repeatfields.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 8e363a0..239fbb5 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -61,6 +61,7 @@ void avfilter_register_all(void) REGISTER_FILTER(APAD, apad, af); REGISTER_FILTER(APERMS, aperms, af); REGISTER_FILTER(APHASER, aphaser, af); + REGISTER_FILTER(AREALTIME, arealtime, af); REGISTER_FILTER(ARESAMPLE, aresample, af); REGISTER_FILTER(AREVERSE, areverse, af); REGISTER_FILTER(ASELECT, aselect, af); @@ -217,6 +218,7 @@ void avfilter_register_all(void) REGISTER_FILTER(PULLUP, pullup, vf); REGISTER_FILTER(QP, qp, vf); REGISTER_FILTER(RANDOM, random, vf); + REGISTER_FILTER(REALTIME, realtime, vf); REGISTER_FILTER(REMOVEGRAIN, removegrain, vf); REGISTER_FILTER(REMOVELOGO, removelogo, vf); REGISTER_FILTER(REPEATFIELDS, repeatfields, vf); diff --git a/libavfilter/f_realtime.c b/libavfilter/f_realtime.c new file mode 100644 index 0000000..a50141c --- /dev/null +++ b/libavfilter/f_realtime.c @@ -0,0 +1,130 @@ +/* + * Copyright (c) 2015 Nicolas George + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/opt.h" +#include "libavutil/time.h" +#include "avfilter.h" +#include "internal.h" + +typedef struct RealtimeContext { + const AVClass *class; + int64_t delta; + int64_t limit; + unsigned inited; +} RealtimeContext; + +static int filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *ctx = inlink->dst; + RealtimeContext *s = ctx->priv; + + if (frame->pts != AV_NOPTS_VALUE) { + int64_t pts = av_rescale_q(frame->pts, inlink->time_base, AV_TIME_BASE_Q); + int64_t now = av_gettime_relative(); + int64_t sleep = pts - now + s->delta; + if (!s->inited) { + s->inited = 1; + sleep = 0; + s->delta = now - pts; + } + if (sleep > s->limit || sleep < -s->limit) { + av_log(ctx, AV_LOG_WARNING, + "time discontinuity detected: %"PRIi64" us, resetting\n", + sleep); + sleep = 0; + s->delta = now - pts; + } + if (sleep > 0) { + av_log(ctx, AV_LOG_DEBUG, "sleeping %"PRIi64" us\n", sleep); + av_usleep(sleep); + } + } + return ff_filter_frame(inlink->dst->outputs[0], frame); +} + +#define OFFSET(x) offsetof(RealtimeContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_FILTERING_PARAM +static const AVOption options[] = { + { "limit", "sleep time limit", OFFSET(limit), AV_OPT_TYPE_DURATION, { .i64 = 2000000 }, 0, INT64_MAX, FLAGS }, + { NULL } +}; + +#if CONFIG_REALTIME_FILTER +#define realtime_options options +AVFILTER_DEFINE_CLASS(realtime); + +static const AVFilterPad avfilter_vf_realtime_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad avfilter_vf_realtime_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_realtime = { + .name = "realtime", + .description = NULL_IF_CONFIG_SMALL("Slow down filtering to match realtime"), + .priv_size = sizeof(RealtimeContext), + .priv_class = &realtime_class, + .inputs = avfilter_vf_realtime_inputs, + .outputs = avfilter_vf_realtime_outputs, +}; +#endif /* CONFIG_REALTIME_FILTER */ + +#if CONFIG_AREALTIME_FILTER + +#define arealtime_options options +AVFILTER_DEFINE_CLASS(arealtime); + +static const AVFilterPad arealtime_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + .filter_frame = filter_frame, + }, + { NULL } +}; + +static const AVFilterPad arealtime_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_AUDIO, + }, + { NULL } +}; + +AVFilter ff_af_arealtime = { + .name = "arealtime", + .description = NULL_IF_CONFIG_SMALL("Slow down filtering to match realtime"), + .priv_size = sizeof(RealtimeContext), + .priv_class = &arealtime_class, + .inputs = arealtime_inputs, + .outputs = arealtime_outputs, +}; +#endif /* CONFIG_AREALTIME_FILTER */ -- 2.6.1

4 8

Evolution of lavfi's design and API
by Nicolas George 07 Jun '18

07 Jun '18

[ CCing Anton, as most that is written here also apply to libav too, and this would be a good occasion to try a cross-fork cooperation; if that is not wanted, please let us know so we can drop the cc. ] 1. Problems with the current design 1.1. Mixed input-/output-driven model Currently, lavfi is designed to work in a mixed input-driven and output-driven model. That means the application needs sometimes to add input to buffersources and sometimes request output to buffersinks. This is a bit of a nuisance, because it requires the application to do it properly: adding input on the wrong input or requesting a frame on the wrong output will cause extra memory consumption or latency. With the libav API, it can not work at all since there is no mechanism to determine which input needs a frame in order to proceed. The libav API is clearly designed for a more output-driven implementation, with FIFOs anywhere to prevent input-driven frames to reach unready filters. Unfortunately, since it is impossible from the outside to guess what output will get a frame next, that can cause frames to accumulate anywhere in the filter graph, eating a lot of memory unnecessarily. FFmpeg's API has eliminated FIFOs in favour of queues in filters that need it, but these queues can not be controlled for unusual filter graphs with extreme needs. Also, there still is an implicit FIFO inside buffersink. 1.2. Recursive implementation All work in a filter graph is triggered by recursive invocations of the filters' methods. It makes debugging harder. It also can lead to large stack usage and makes frame- and filter-level multithreading harder to implement. It also prevents some diagnosis from working reliably. 1.3. EOF handling Currently, EOF is propagated only through the return value of the request_frame() method. That means it only works in an output-driven scheme. It also means that it has no timestamp attached to it; this is an issue for filters where the duration of the last frame is relevant, like vf_fps. 1.4. Latency Some filters need to know the timestamp of the next frame in order to know when the current frame will stop and be able to process it: overlay, fps are two examples. These filters will introduce a latency of one input frame that could otherwise be avoided. 1.5. Timestamps Some filters do not care about timestamps at all. Some check and have a proper handling of NOPTS values. Some filters just assume the frames will have timestamps, and possibly make extra assumptions on that: monotony, consistency, etc. That is an inconsistent mess. 1.6. Sparse streams There is a more severe instance of the latency issue when the input comes from an interleaved sparse stream: in that case, waiting for the next frame in order to find the end of the current one may require demuxing a large chunk of input, in turn provoking a lot of activity on other inputs of the graph. 2. Proposed API changes To fix/enhance all these issues, I believe a complete rethink of the scheduling design of the library is necessary. I propose the following changes. Note: some of these changes are not 100% related to the issues I raised, but looked like a good idea while thinking on an API rework. 2.1. AVFrame.duration Add a duration field to AVFrame; if set, it indicates the duration of the frame. Thus, it becomes unnecessary to wait for the next frame to know when the current frame stops, reducing the latency. Another solution would be to add a dedicated function on buffersrc to inject a timestamp for end or activity on a link. That would avoid the need of adding a field to AVFrame. 2.2. Add some fields to AVFilterLink AVFilterLink.pts: current timestamp of the link, i.e. end timestamp of the last forwarede frame, assuming the duration was correct. This is somewhat redundant with the fields in AVFrame, but can carry the information even when there is no actual frame. AVFilterLink.status: if not 0, gives the return status of trying to pass a frame on this link. The typical use would be EOF. 2.3. AVFilterLink.need_ts Add a field to AVFilterLink to specify that the output filter requires reliable timestamps on its input. More precisely, specify how reliable the timestamps need to be: is the duration necessary? do the timestamps need to be monotonic? continuous? For audio streams, consistency between timestamps and the number of samples may also be tested. For video streams, constant frame rate may be enforced, but I am not sure about this one. A "fixpts" filter should be provided to allow the user to tweak how the timestamps are fixed (change the timestamps to match the duration or change the duration to match the timestamps?). When no explicit filter is inserted, the framework should do the work of fixing them automatically. I am not sure whether that should be done directly or by automatically inserting the fixpts filter. The later solution is more elegant, but it requires more changes to the framework and the filters (because the correctness of the timestamps would need to be merged just like formats), so I am rather for the former. Note that for a lot of filters, the actual duration or end timestamp is not required, only a lower bound for it. For sparse interleaved streams, that is very relevant as we may not know the exact time for the next frame until we reach it, but we can know it is later than the other streams' timestamps minus the interleaving delta. 2.4. Build FIFOs directly in AVFilterLink Instead of automatically insert an additional filter like libav, handle the FIFO operation directly in the framework using fields in AVFilterLink. The main benefit is that the framework can examine the inside of the FIFOs to make scheduling decisions. It can also do so to provide the user with more accurate diagnostics. An extra benefit: the memory pool for the FIFOed frames can more easily be shared, across the whole filter graph or the whole application. Memory management becomes easier: just take a good heuristics (half the RAM?), no need to guess what FIFOs will actually need a lot of memory and what FIFOs are just there mostly useless. Last but not least, FIFOs now become potential thread communication / synchronization points, making filter-level multithreading easier. For audio streams, framing (i.e. ensuring all frame have an exact / minimum / maximum number of samples) can be merged with FIFOs. 2.5. Allow status change pseudo-frames inside FIFOs To propagate EOF and possibly other status changes (errors) in input-driven model, allow FIFOs to contain not only frames but also kind of pseudo-frames with a timestamp and metadata attached. Depending on the filters, these pseudo-frames may be directly passed to the filter's methods, or they may be interpreted by the framework to just change fields on the AVFilterLink structure. 2.6. Change the scheduling logic for filters From the outside of a filter with several outputs, it is usually not possible to guess what output will get a frame next. Requesting a frame on output #0 may cause activity on the filter graph that produce a frame on output #1 instead, or possibly on a completely different filter. Therefore, having a request_frame() method on all outputs seems pointless. Instead, use a global AVFilter.activate() method that causes the filter to do one step of work if it can. This method is called each time something is changed to the filter: new frame on input, output ready, status change. It returns as soon at it could do something, either producing output and/or consuming input, or nothing if nothing can be done. 2.7. Add fields to AVFilterLink for flow control. Add to AVFilterLinks a few field to help filters decide if they need to process something, and if relevant in what order. The most obvious idea would be AVFilterLink.frames_needed, counting how many frames are probably needed on a link before anything can be done. For example, with concat, after input has been consumed, the frames_needed fields on the current input are set according to the corresponding output. 2.8. Activate the filters iteratively Keep a global (per graph) priority queue of filters that are supposed to be ready and call the activate() method on them. 2.9. AVFrame.stream_id Add an integer (or pointer: intptr_t maybe?) field to AVFrame to allow passing frames related to distinct streams on the same link. That would allow to multiplex all outputs of a graph into a single output, making the application simpler. Not sure this is really useful or necessary: for the graph outputs, a convenience function iterating on all of them and returning the frame and the output index separately would do the trick too. 2.10. buffersrc.callback and buffersink.callback Add a callback on both buffersource and buffersink, called respectively when a frame is necessary on input and a frame has arrived on output. This allows pure input-driven and pure output-driven design to work. 2.11. Links groups Links that carry frames from related interleaved streams should be explicitly connected together so that the framework can use the information. The typical use would be to group all the links from buffersrc that come from the same interleaved input file. When a frame is passed on a link, all links in the same group(s) that are too late (according to an interleaving tolerance that can be set) are activated using a dummy frame. 2.12. FIFOs with compression and external storage All FIFOs should be able to off-load some of their memory requirements by either compressing the frames (using a lossless or optionally lossy codec) and/or storing them on mass storage. The options for that should be changeable globally or on a per-link basis. 2.13. AVFrame.owner Add a owner field (probably with type "AVObject", i.e. "void *" pointing to AVClass *) to AVFrame, and update it whenever the frame is passed from one filter to the other. That way, inconsistent ref/unref operations can be detected. Regards, -- Nicolas George

7 35

FFmpeg at Chemnitzer Linux-Tage 2016
by Thilo Borgmann 31 Jan '18

31 Jan '18

Hi, I just registered FFmpeg for participation at the annual Chemnitzer Linux-Tage in 2016 (CLT) which takes place in Chemnitz, Germany from the 19th to the 20th March 2016. Like in the past years, we are planning to man a booth with some of our developers, give direct user support and receiving valuable feedback and non-working samples from our users. The registration is ongoing until early January and still has to be confirmed by the organization of the CLT. Currently, the following developers are registered for our booth: Carl Eugen Hoyos Thomas Volkert Thilo Borgmann For the last years, Alexander Strasser did all the administration and most of the organization. This year, I volunteered to handle all this stuff. If you're a developer and interested to participate at and/or visit our booth, we are happy to see you in Chemnitz! Feel free to contact me about anything regarding this. Like in the past years, chances are good that we can compensate travel and hotel costs for our booth staff developers. Once our registration is confirmed or declined, I will report back about it and if we are accepted for participation, I will also post on FFmpeg-user to announce our attendance. -Thilo

1 3

[PATCH 1/2] avcodec/h264: keep SPS and PPS bitstream data
by wm4 26 Sep '17

26 Sep '17

Needed for the following VideotoolBox commit. --- libavcodec/h264.h | 4 ++++ libavcodec/h264_ps.c | 20 ++++++++++++++++---- 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/libavcodec/h264.h b/libavcodec/h264.h index 7356288..eeb2aaf 100644 --- a/libavcodec/h264.h +++ b/libavcodec/h264.h @@ -229,6 +229,8 @@ typedef struct SPS { int residual_color_transform_flag; ///< residual_colour_transform_flag int constraint_set_flags; ///< constraint_set[0-3]_flag int new; ///< flag to keep track if the decoder context needs re-init due to changed SPS + uint8_t data[1 << 16]; + int data_size; } SPS; /** @@ -254,6 +256,8 @@ typedef struct PPS { uint8_t scaling_matrix8[6][64]; uint8_t chroma_qp_table[2][QP_MAX_NUM+1]; ///< pre-scaled (with chroma_qp_index_offset) version of qp_table int chroma_qp_diff; + uint8_t data[1 << 16]; + int data_size; } PPS; /** diff --git a/libavcodec/h264_ps.c b/libavcodec/h264_ps.c index 52d235c..fd16a95 100644 --- a/libavcodec/h264_ps.c +++ b/libavcodec/h264_ps.c @@ -307,6 +307,15 @@ int ff_h264_decode_seq_parameter_set(H264Context *h, int ignore_truncation) int i, log2_max_frame_num_minus4; SPS *sps; + sps = av_mallocz(sizeof(SPS)); + if (!sps) + return AVERROR(ENOMEM); + + sps->data_size = h->gb.buffer_end - h->gb.buffer; + if (sps->data_size > sizeof(sps->data)) + goto fail; + memcpy(sps->data, h->gb.buffer, sps->data_size); + profile_idc = get_bits(&h->gb, 8); constraint_set_flags |= get_bits1(&h->gb) << 0; // constraint_set0_flag constraint_set_flags |= get_bits1(&h->gb) << 1; // constraint_set1_flag @@ -320,11 +329,8 @@ int ff_h264_decode_seq_parameter_set(H264Context *h, int ignore_truncation) if (sps_id >= MAX_SPS_COUNT) { av_log(h->avctx, AV_LOG_ERROR, "sps_id %u out of range\n", sps_id); - return AVERROR_INVALIDDATA; + goto fail; } - sps = av_mallocz(sizeof(SPS)); - if (!sps) - return AVERROR(ENOMEM); sps->sps_id = sps_id; sps->time_offset_length = 24; @@ -603,6 +609,12 @@ int ff_h264_decode_picture_parameter_set(H264Context *h, int bit_length) pps = av_mallocz(sizeof(PPS)); if (!pps) return AVERROR(ENOMEM); + pps->data_size = h->gb.buffer_end - h->gb.buffer; + if (pps->data_size > sizeof(pps->data)) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + memcpy(pps->data, h->gb.buffer, pps->data_size); pps->sps_id = get_ue_golomb_31(&h->gb); if ((unsigned)pps->sps_id >= MAX_SPS_COUNT || !h->sps_buffers[pps->sps_id]) { -- 2.5.1

4 15

[PATCH] lavf/mov: add support for sidx fragment indexes
by Rodger Combs 18 Jul '17

18 Jul '17

Fixes trac #3842 --- libavformat/isom.h | 2 + libavformat/mov.c | 245 ++++++++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 208 insertions(+), 39 deletions(-) diff --git a/libavformat/isom.h b/libavformat/isom.h index aee9d6e..6e921c0 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -103,6 +103,7 @@ typedef struct MOVSbgp { typedef struct MOVFragmentIndexItem { int64_t moof_offset; int64_t time; + int headers_read; } MOVFragmentIndexItem; typedef struct MOVFragmentIndex { @@ -197,6 +198,7 @@ typedef struct MOVContext { int has_looked_for_mfra; MOVFragmentIndex** fragment_index_data; unsigned fragment_index_count; + int fragment_index_complete; int atom_depth; unsigned int aax_mode; ///< 'aax' file has been detected uint8_t file_key[20]; diff --git a/libavformat/mov.c b/libavformat/mov.c index da170a6..8463d57 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -3349,7 +3349,7 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) MOVFragment *frag = &c->fragment; MOVTrackExt *trex = NULL; MOVFragmentIndex* index = NULL; - int flags, track_id, i; + int flags, track_id, i, found = 0; avio_r8(pb); /* version */ flags = avio_rb24(pb); @@ -3367,15 +3367,6 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) av_log(c->fc, AV_LOG_ERROR, "could not find corresponding trex\n"); return AVERROR_INVALIDDATA; } - for (i = 0; i < c->fragment_index_count; i++) { - MOVFragmentIndex* candidate = c->fragment_index_data[i]; - if (candidate->track_id == frag->track_id) { - av_log(c->fc, AV_LOG_DEBUG, - "found fragment index for track %u\n", frag->track_id); - index = candidate; - break; - } - } frag->base_data_offset = flags & MOV_TFHD_BASE_DATA_OFFSET ? avio_rb64(pb) : flags & MOV_TFHD_DEFAULT_BASE_IS_MOOF ? @@ -3389,24 +3380,33 @@ static int mov_read_tfhd(MOVContext *c, AVIOContext *pb, MOVAtom atom) frag->flags = flags & MOV_TFHD_DEFAULT_FLAGS ? avio_rb32(pb) : trex->flags; frag->time = AV_NOPTS_VALUE; - if (index) { - int i, found = 0; - for (i = index->current_item; i < index->item_count; i++) { - if (frag->implicit_offset == index->items[i].moof_offset) { - av_log(c->fc, AV_LOG_DEBUG, "found fragment index entry " - "for track %u and moof_offset %"PRId64"\n", - frag->track_id, index->items[i].moof_offset); - frag->time = index->items[i].time; - index->current_item = i + 1; - found = 1; + for (i = 0; i < c->fragment_index_count; i++) { + int j; + MOVFragmentIndex* candidate = c->fragment_index_data[i]; + if (candidate->track_id == frag->track_id) { + av_log(c->fc, AV_LOG_DEBUG, + "found fragment index for track %u\n", frag->track_id); + index = candidate; + for (j = index->current_item; j < index->item_count; j++) { + if (frag->implicit_offset == index->items[j].moof_offset) { + av_log(c->fc, AV_LOG_DEBUG, "found fragment index entry " + "for track %u and moof_offset %"PRId64"\n", + frag->track_id, index->items[j].moof_offset); + frag->time = index->items[j].time; + index->current_item = j + 1; + found = 1; + break; + } } - } - if (!found) { - av_log(c->fc, AV_LOG_WARNING, "track %u has a fragment index " - "but it doesn't have an (in-order) entry for moof_offset " - "%"PRId64"\n", frag->track_id, frag->implicit_offset); + if (found) + break; } } + if (index && !found) { + av_log(c->fc, AV_LOG_DEBUG, "track %u has a fragment index but " + "it doesn't have an (in-order) entry for moof_offset " + "%"PRId64"\n", frag->track_id, frag->implicit_offset); + } av_log(c->fc, AV_LOG_TRACE, "frag flags 0x%x\n", frag->flags); return 0; } @@ -3596,7 +3596,106 @@ static int mov_read_trun(MOVContext *c, AVIOContext *pb, MOVAtom atom) return AVERROR_EOF; frag->implicit_offset = offset; - st->duration = sc->track_end = dts + sc->time_offset; + + sc->track_end = dts + sc->time_offset; + if (st->duration < sc->track_end) + st->duration = sc->track_end; + + return 0; +} + +static int mov_read_sidx(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + int64_t offset = avio_tell(pb) + atom.size, pts; + uint8_t version; + unsigned i, track_id; + AVStream *st = NULL; + MOVStreamContext *sc; + MOVFragmentIndex *index = NULL; + MOVFragmentIndex **tmp; + AVRational timescale; + + version = avio_r8(pb); + if (version > 1) { + avpriv_request_sample(c->fc, "sidx version %u", version); + return AVERROR_PATCHWELCOME; + } + + avio_rb24(pb); // flags + + track_id = avio_rb32(pb); // Reference ID + for (i = 0; i < c->fc->nb_streams; i++) { + if (c->fc->streams[i]->id == track_id) { + st = c->fc->streams[i]; + break; + } + } + if (!st) { + av_log(c->fc, AV_LOG_ERROR, "could not find corresponding track id %d\n", track_id); + return AVERROR_INVALIDDATA; + } + + sc = st->priv_data; + + timescale = av_make_q(1, avio_rb32(pb)); + + if (version == 0) { + pts = avio_rb32(pb); + offset += avio_rb32(pb); + } else { + pts = avio_rb64(pb); + offset += avio_rb64(pb); + } + + avio_rb16(pb); // reserved + + index = av_mallocz(sizeof(MOVFragmentIndex)); + if (!index) + return AVERROR(ENOMEM); + + index->track_id = track_id; + + index->item_count = avio_rb16(pb); + index->items = av_mallocz_array(index->item_count, sizeof(MOVFragmentIndexItem)); + + if (!index->items) { + av_freep(&index); + return AVERROR(ENOMEM); + } + + for (i = 0; i < index->item_count; i++) { + uint32_t size = avio_rb32(pb); + uint32_t duration = avio_rb32(pb); + if (size & 0x80000000) { + avpriv_request_sample(c->fc, "sidx reference_type 1"); + av_freep(&index->items); + av_freep(&index); + return AVERROR_PATCHWELCOME; + } + avio_rb32(pb); // sap_flags + index->items[i].moof_offset = offset; + index->items[i].time = av_rescale_q(pts, st->time_base, timescale); + offset += size; + pts += duration; + } + + st->duration = sc->track_end = pts; + + tmp = av_realloc_array(c->fragment_index_data, + c->fragment_index_count + 1, + sizeof(MOVFragmentIndex*)); + if (!tmp) { + av_freep(&index->items); + av_freep(&index); + return AVERROR(ENOMEM); + } + + c->fragment_index_data = tmp; + c->fragment_index_data[c->fragment_index_count++] = index; + + if (offset == avio_size(pb)) + c->fragment_index_complete = 1; + return 0; } @@ -3854,6 +3953,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('a','l','a','c'), mov_read_alac }, /* alac specific atom */ { MKTAG('a','v','c','C'), mov_read_glbl }, { MKTAG('p','a','s','p'), mov_read_pasp }, +{ MKTAG('s','i','d','x'), mov_read_sidx }, { MKTAG('s','t','b','l'), mov_read_default }, { MKTAG('s','t','c','o'), mov_read_stco }, { MKTAG('s','t','p','s'), mov_read_stps }, @@ -3978,9 +4078,9 @@ static int mov_read_default(MOVContext *c, AVIOContext *pb, MOVAtom atom) return err; } if (c->found_moov && c->found_mdat && - ((!pb->seekable || c->fc->flags & AVFMT_FLAG_IGNIDX) || + ((!pb->seekable || c->fc->flags & AVFMT_FLAG_IGNIDX || c->fragment_index_complete) || start_pos + a.size == avio_size(pb))) { - if (!pb->seekable || c->fc->flags & AVFMT_FLAG_IGNIDX) + if (!pb->seekable || c->fc->flags & AVFMT_FLAG_IGNIDX || c->fragment_index_complete) c->next_root_atom = start_pos + a.size; c->atom_depth --; return 0; @@ -4585,6 +4685,52 @@ static int should_retry(AVIOContext *pb, int error_code) { return 1; } +static int mov_switch_root(AVFormatContext *s, int64_t target) +{ + MOVContext *mov = s->priv_data; + int i, j; + int already_read = 0; + + if (avio_seek(s->pb, target, SEEK_SET) != target) { + av_log(mov->fc, AV_LOG_ERROR, "root atom offset 0x%"PRIx64": partial file\n", target); + return AVERROR_INVALIDDATA; + } + + mov->next_root_atom = 0; + + for (i = 0; i < mov->fragment_index_count; i++) { + MOVFragmentIndex *index = mov->fragment_index_data[i]; + int found = 0; + for (j = 0; j < index->item_count; j++) { + MOVFragmentIndexItem *item = &index->items[j]; + if (found) { + mov->next_root_atom = item->moof_offset; + break; // Advance to next index in outer loop + } else if (item->moof_offset == target) { + index->current_item = FFMIN(j, index->current_item); + if (item->headers_read) + already_read = 1; + item->headers_read = 1; + found = 1; + } + } + if (!found) + index->current_item = 0; + } + + if (already_read) + return 0; + + mov->found_mdat = 0; + + if (mov_read_default(mov, s->pb, (MOVAtom){ AV_RL32("root"), INT64_MAX }) < 0 || + avio_feof(s->pb)) + return AVERROR_EOF; + av_log(s, AV_LOG_TRACE, "read fragments, offset 0x%"PRIx64"\n", avio_tell(s->pb)); + + return 1; +} + static int mov_read_packet(AVFormatContext *s, AVPacket *pkt) { MOVContext *mov = s->priv_data; @@ -4595,19 +4741,11 @@ static int mov_read_packet(AVFormatContext *s, AVPacket *pkt) mov->fc = s; retry: sample = mov_find_next_sample(s, &st); - if (!sample) { - mov->found_mdat = 0; + if (!sample || (mov->next_root_atom && sample->pos > mov->next_root_atom)) { if (!mov->next_root_atom) return AVERROR_EOF; - if (avio_seek(s->pb, mov->next_root_atom, SEEK_SET) != mov->next_root_atom) { - av_log(mov->fc, AV_LOG_ERROR, "next root atom offset 0x%"PRIx64": partial file\n", mov->next_root_atom); - return AVERROR_INVALIDDATA; - } - mov->next_root_atom = 0; - if (mov_read_default(mov, s->pb, (MOVAtom){ AV_RL32("root"), INT64_MAX }) < 0 || - avio_feof(s->pb)) - return AVERROR_EOF; - av_log(s, AV_LOG_TRACE, "read fragments, offset 0x%"PRIx64"\n", avio_tell(s->pb)); + if ((ret = mov_switch_root(s, mov->next_root_atom)) < 0) + return ret; goto retry; } sc = st->priv_data; @@ -4685,12 +4823,41 @@ static int mov_read_packet(AVFormatContext *s, AVPacket *pkt) return 0; } +static int mov_seek_fragment(AVFormatContext *s, AVStream *st, int64_t timestamp) +{ + MOVContext *mov = s->priv_data; + int i, j; + + if (!mov->fragment_index_complete) + return 0; + + for (i = 0; i < mov->fragment_index_count; i++) { + if (mov->fragment_index_data[i]->track_id == st->id) { + MOVFragmentIndex *index = index = mov->fragment_index_data[i]; + for (j = index->item_count - 1; j >= 0; j--) { + if (index->items[j].time <= timestamp) { + if (index->items[j].headers_read) + return 0; + + return mov_switch_root(s, index->items[j].moof_offset); + } + } + } + } + + return 0; +} + static int mov_seek_stream(AVFormatContext *s, AVStream *st, int64_t timestamp, int flags) { MOVStreamContext *sc = st->priv_data; int sample, time_sample; int i; + int ret = mov_seek_fragment(s, st, timestamp); + if (ret < 0) + return ret; + sample = av_index_search_timestamp(st, timestamp, flags); av_log(s, AV_LOG_TRACE, "stream %d, timestamp %"PRId64", sample %d\n", st->index, timestamp, sample); if (sample < 0 && st->nb_index_entries && timestamp < st->index_entries[0].timestamp) -- 2.6.0

4 11

[PATCH] checkasm: add fixed_dsp tests
by James Almer 11 Apr '17

11 Apr '17

Signed-off-by: James Almer <jamrial(a)gmail.com> --- tests/checkasm/Makefile | 3 + tests/checkasm/checkasm.c | 3 + tests/checkasm/checkasm.h | 1 + tests/checkasm/fixed_dsp.c | 169 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 176 insertions(+) create mode 100644 tests/checkasm/fixed_dsp.c diff --git a/tests/checkasm/Makefile b/tests/checkasm/Makefile index e38d15b..698ceef 100644 --- a/tests/checkasm/Makefile +++ b/tests/checkasm/Makefile @@ -11,7 +11,10 @@ AVCODECOBJS-$(CONFIG_PIXBLOCKDSP) += pixblockdsp.o AVCODECOBJS-$(CONFIG_V210_ENCODER) += v210enc.o AVCODECOBJS-$(CONFIG_VP9_DECODER) += vp9dsp.o +AVUTILOBJS += fixed_dsp.o + CHECKASMOBJS-$(CONFIG_AVCODEC) += $(AVCODECOBJS-yes) +CHECKASMOBJS-$(CONFIG_AVUTIL) += $(AVUTILOBJS) -include $(SRC_PATH)/tests/checkasm/$(ARCH)/Makefile diff --git a/tests/checkasm/checkasm.c b/tests/checkasm/checkasm.c index 7f2cf8e..1da085c 100644 --- a/tests/checkasm/checkasm.c +++ b/tests/checkasm/checkasm.c @@ -58,6 +58,9 @@ static const struct { const char *name; void (*func)(void); } tests[] = { +#if CONFIG_AVUTIL + { "fixed_dsp", checkasm_check_fixed_dsp }, +#endif #if CONFIG_AVCODEC #if CONFIG_ALAC_DECODER { "alacdsp", checkasm_check_alacdsp }, diff --git a/tests/checkasm/checkasm.h b/tests/checkasm/checkasm.h index 69416e9..dc3519d 100644 --- a/tests/checkasm/checkasm.h +++ b/tests/checkasm/checkasm.h @@ -33,6 +33,7 @@ void checkasm_check_alacdsp(void); void checkasm_check_bswapdsp(void); void checkasm_check_dcadsp(void); +void checkasm_check_fixed_dsp(void); void checkasm_check_flacdsp(void); void checkasm_check_fmtconvert(void); void checkasm_check_h264pred(void); diff --git a/tests/checkasm/fixed_dsp.c b/tests/checkasm/fixed_dsp.c new file mode 100644 index 0000000..ac7a0b4 --- /dev/null +++ b/tests/checkasm/fixed_dsp.c @@ -0,0 +1,169 @@ +/* + * Copyright (c) 2015 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with FFmpeg; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ + +#include "checkasm.h" +#include "libavutil/common.h" +#include "libavutil/fixed_dsp.h" +#include "libavutil/internal.h" +#include "libavutil/intreadwrite.h" +#include "libavutil/mem.h" + +#define BUF_SIZE 256 + +#define randomize_buffers() \ + do { \ + int i; \ + for (i = 0; i < BUF_SIZE; i++) { \ + src0[i] = sign_extend(rnd(), 24); \ + src1[i] = sign_extend(rnd(), 24); \ + src2[i] = sign_extend(rnd(), 24); \ + } \ + } while (0) + +static void check_vector_fmul(const int *src0, const int *src1, const char *name) +{ + LOCAL_ALIGNED_32(int, ref, [BUF_SIZE]); + LOCAL_ALIGNED_32(int, new, [BUF_SIZE]); + + declare_func(void, int *dst, const int *src0, const int *src1, int len); + + call_ref(ref, src0, src1, BUF_SIZE); + call_new(new, src0, src1, BUF_SIZE); + if (memcmp(ref, new, BUF_SIZE * sizeof(int))) + fail(); + bench_new(new, src0, src1, BUF_SIZE); + + report(name); +} + +static void check_vector_fmul_window_scaled(const int32_t *src0, const int32_t *src1, const int32_t *win) +{ + LOCAL_ALIGNED_16(int16_t, ref, [BUF_SIZE]); + LOCAL_ALIGNED_16(int16_t, new, [BUF_SIZE]); + + declare_func(void, int16_t *dst, const int32_t *src0, const int32_t *src1, const int32_t *win, int len, uint8_t bits); + + call_ref(ref, src0, src1, win, BUF_SIZE / 2, 2); + call_new(new, src0, src1, win, BUF_SIZE / 2, 2); + if (memcmp(ref, new, BUF_SIZE * sizeof(int16_t))) + fail(); + bench_new(new, src0, src1, win, BUF_SIZE / 2, 2); + + report("vector_fmul_window_scaled"); +} + +static void check_vector_fmul_window(const int32_t *src0, const int32_t *src1, const int32_t *win) +{ + LOCAL_ALIGNED_32(int32_t, ref, [BUF_SIZE]); + LOCAL_ALIGNED_32(int32_t, new, [BUF_SIZE]); + + declare_func(void, int32_t *dst, const int32_t *src0, const int32_t *src1, const int32_t *win, int len); + + call_ref(ref, src0, src1, win, BUF_SIZE / 2); + call_new(new, src0, src1, win, BUF_SIZE / 2); + if (memcmp(ref, new, BUF_SIZE * sizeof(int32_t))) + fail(); + bench_new(new, src0, src1, win, BUF_SIZE / 2); + + report("vector_fmul_window"); +} + +static void check_vector_fmul_add(const int *src0, const int *src1, const int *src2) +{ + LOCAL_ALIGNED_32(int, ref, [BUF_SIZE]); + LOCAL_ALIGNED_32(int, new, [BUF_SIZE]); + + declare_func(void, int *dst, const int *src0, const int *src1, const int *src2, int len); + + call_ref(ref, src0, src1, src2, BUF_SIZE); + call_new(new, src0, src1, src2, BUF_SIZE); + if (memcmp(ref, new, BUF_SIZE * sizeof(int))) + fail(); + bench_new(new, src0, src1, src2, BUF_SIZE); + + report("vector_fmul_add"); +} + +static void check_butterflies(const int *src0, const int *src1) +{ + LOCAL_ALIGNED_16(int, ref0, [BUF_SIZE]); + LOCAL_ALIGNED_16(int, ref1, [BUF_SIZE]); + LOCAL_ALIGNED_16(int, new0, [BUF_SIZE]); + LOCAL_ALIGNED_16(int, new1, [BUF_SIZE]); + + declare_func(void, int *av_restrict src0, int *av_restrict src1, int len); + + memcpy(ref0, src0, BUF_SIZE * sizeof(*src0)); + memcpy(ref1, src1, BUF_SIZE * sizeof(*src1)); + memcpy(new0, src0, BUF_SIZE * sizeof(*src0)); + memcpy(new1, src1, BUF_SIZE * sizeof(*src1)); + + call_ref(ref0, ref1, BUF_SIZE); + call_new(new0, new1, BUF_SIZE); + if (memcmp(ref0, new0, BUF_SIZE * sizeof(*ref0)) || + memcmp(ref1, new1, BUF_SIZE * sizeof(*ref1))) + fail(); + memcpy(new0, src0, BUF_SIZE * sizeof(*src0)); + memcpy(new1, src1, BUF_SIZE * sizeof(*src1)); + bench_new(new0, new1, BUF_SIZE); + + report("butterflies_fixed"); +} + +static void check_scalarproduct_fixed(const int *src0, const int *src1) +{ + int ref, new; + + declare_func(int, const int *src0, const int *src1, int len); + + ref = call_ref(src0, src1, BUF_SIZE); + new = call_new(src0, src1, BUF_SIZE); + if (ref != new) + fail(); + bench_new(src0, src1, BUF_SIZE); + + report("scalarproduct_fixed"); +} + +void checkasm_check_fixed_dsp(void) +{ + LOCAL_ALIGNED_32(int32_t, src0, [BUF_SIZE]); + LOCAL_ALIGNED_32(int32_t, src1, [BUF_SIZE]); + LOCAL_ALIGNED_32(int32_t, src2, [BUF_SIZE]); + AVFixedDSPContext *fdsp = avpriv_alloc_fixed_dsp(1); + + randomize_buffers(); + if (check_func(fdsp->vector_fmul, "vector_fmul")) + check_vector_fmul(src0, src1, "vector_fmul"); + if (check_func(fdsp->vector_fmul_reverse, "vector_fmul_reverse")) + check_vector_fmul(src0, src1, "vector_fmul_reverse"); + if (check_func(fdsp->vector_fmul_window_scaled, "vector_fmul_window_scaled")) + check_vector_fmul_window_scaled(src0, src1, src2); + if (check_func(fdsp->vector_fmul_window, "vector_fmul_window")) + check_vector_fmul_window(src0, src1, src2); + if (check_func(fdsp->vector_fmul_add, "vector_fmul_add")) + check_vector_fmul_add(src0, src1, src2); + if (check_func(fdsp->butterflies_fixed, "butterflies_fixed")) + check_butterflies(src0, src1); + if (check_func(fdsp->scalarproduct_fixed, "scalarproduct_fixed")) + check_scalarproduct_fixed(src0, src1); + + av_freep(&fdsp); +} -- 2.7.0

3 5

[PATCH]lavf/mpeg: Identify sub-stream ID 0xa1 as mlp
by Carl Eugen Hoyos 02 Mar '17

02 Mar '17

Hi! Attached patch (still) fixes ticket #4786 for me. Autodetection seems impossible to me and the following page suggests that this patch is likely correct: http://dvd-audio.sourceforge.net/spec/aob.shtml Please comment, Carl Eugen

3 3

[rfc] ffmpeg security issue mailing list
by compn 09 Feb '17

09 Feb '17

hello, some of you know that we have a list for security / CVE issues. some of you did not know this. i think it is a private list due to not wanting people to make exploits before we have a chance to fix them. of course, if no one is subscribed to review/fix issues then they will never get fixed. so if you are a regular developer who wants access to this list, please speak up. i do not run nor admin the security email/list (nor do i know who does) so please dont ask me questions about it. -compn

6 12

[PATCH] avfilter: add threshold filter
by Paul B Mahol 30 Jan '17

30 Jan '17

Hi, patch attached.

1 1