August 2022 - ffmpeg-devel - lists.ffmpeg.org

[PATCH 0/8] rewrite avpriv_find_start_code() for clarity
by Scott Theisen 16 Sep '22

16 Sep '22

I am endeavoring to reduce MythTV’s downstream changes to FFmpeg. avpriv_find_start_code() is one of the internal FFmpeg functions that MythTV uses. I was planning on copying it into MythTV to eventually eliminate all uses of internal FFmpeg headers; however, what avpriv_find_start_code() actually does was not very clear. Therefore, I rewrote it and added comments to make its purpose and effects clearer.

2 62

[PATCH] lavu: make av_get_media_type_string() never return NULL
by Scott Theisen 16 Sep '22

16 Sep '22

printf %s with a null pointer is undefined behavior --- libavutil/avutil.h | 3 +-- libavutil/utils.c | 3 ++- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/libavutil/avutil.h b/libavutil/avutil.h index 4d633156d1..4bd468d72f 100644 --- a/libavutil/avutil.h +++ b/libavutil/avutil.h @@ -207,8 +207,7 @@ enum AVMediaType { }; /** - * Return a string describing the media_type enum, NULL if media_type - * is unknown. + * Return a string describing the media_type enum, never NULL. */ const char *av_get_media_type_string(enum AVMediaType media_type); diff --git a/libavutil/utils.c b/libavutil/utils.c index ea9b5097b8..c85d7abace 100644 --- a/libavutil/utils.c +++ b/libavutil/utils.c @@ -71,12 +71,13 @@ const char *avutil_license(void) const char *av_get_media_type_string(enum AVMediaType media_type) { switch (media_type) { + case AVMEDIA_TYPE_UNKNOWN: return "unknown"; case AVMEDIA_TYPE_VIDEO: return "video"; case AVMEDIA_TYPE_AUDIO: return "audio"; case AVMEDIA_TYPE_DATA: return "data"; case AVMEDIA_TYPE_SUBTITLE: return "subtitle"; case AVMEDIA_TYPE_ATTACHMENT: return "attachment"; - default: return NULL; + default: return "invalid"; } } -- 2.32.0

4 18

[PATCH] fftools/ffplay: fix rotation incorrect when frame contains the side_data(type is AV_FRAME_DATA_DISPLAYMATRIX)
by 1035567130＠qq.com 15 Sep '22

15 Sep '22

From: Wang Yaqiang <wangyaqiang03(a)kuaishou.com> For example, if the jpeg contains exif information and the rotation direction is included in the exif, the displaymatrix will be set on the side_data of the frame when decoding. However, when ffplay is used to play the image, only the side data in the stream will be determined. It does not check whether the frame also contains rotation information, causing it to play in the wrong direction --- fftools/ffplay.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/fftools/ffplay.c b/fftools/ffplay.c index 9242047f5c..868123dc65 100644 --- a/fftools/ffplay.c +++ b/fftools/ffplay.c @@ -1917,7 +1917,8 @@ static int configure_video_filters(AVFilterGraph *graph, VideoState *is, const c if (autorotate) { int32_t *displaymatrix = (int32_t *)av_stream_get_side_data(is->video_st, AV_PKT_DATA_DISPLAYMATRIX, NULL); double theta = get_rotation(displaymatrix); - + int frame_rotation_checked = 0; + rotation: if (fabs(theta - 90) < 1.0) { INSERT_FILT("transpose", "clock"); } else if (fabs(theta - 180) < 1.0) { @@ -1930,6 +1931,15 @@ static int configure_video_filters(AVFilterGraph *graph, VideoState *is, const c snprintf(rotate_buf, sizeof(rotate_buf), "%f*PI/180", theta); INSERT_FILT("rotate", rotate_buf); } + if (!frame_rotation_checked) { + frame_rotation_checked = 1; + AVFrameSideData *sd = av_frame_get_side_data(frame,AV_FRAME_DATA_DISPLAYMATRIX); + if (sd) { + displaymatrix = (int32_t *)sd->data; + theta = get_rotation(displaymatrix); + goto rotation; + } + } } if ((ret = configure_filtergraph(graph, vfilters, filt_src, last_filter)) < 0) -- 2.33.0

5 11

[PATCH 1/7] avcodec/iff: Split extract_header into extradata and packet part
by Andreas Rheinhardt 15 Sep '22

15 Sep '22

183132872a1d8bc8a32e7fd8f994fa2f1b2d6bfc made the iff demuxer output extradata and made the decoder parse said extradata. To make this extradata extensible, it came with its own internal length field (containing the offset of the palette at the end of the extradata). Furthermore, in order to support mid-stream extradata changes, the packets returned by the demuxer also have such a length field (containing the offset of the actual packet data). Therefore the packet parsing the extradata accepted its input from both AVPackets as well as from ordinary extradata. Yet the demuxer never made use of this "feature": The packet's length field always indicated that the packet data starts immediately after the length field. Later, commit cb928fc448f9566e6f6c28d53fa4c2388e732a2b stopped appending the length field to the packets' data; of course, it also stopped searching for extradata in this data. Instead it added code to parse the packet's header to the function that parses extradata. This made this function consist of two disjoint parts, one of which is only reachable if this function is called from init (when parsing extradata) and one of which is reachable when parsing packet headers. Therefore this commit splits this function into two. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt(a)outlook.com> --- Btw: Both of these commits broke API/ABI. libavcodec/iff.c | 116 +++++++++++++++++++++++++---------------------- 1 file changed, 61 insertions(+), 55 deletions(-) diff --git a/libavcodec/iff.c b/libavcodec/iff.c index 74ebddc621..4abbed8dfb 100644 --- a/libavcodec/iff.c +++ b/libavcodec/iff.c @@ -199,11 +199,9 @@ static int cmap_read_palette(AVCodecContext *avctx, uint32_t *pal) * decoder structures. * * @param avctx the AVCodecContext where to extract extra context to - * @param avpkt the AVPacket to extract extra context from or NULL to use avctx * @return >= 0 in case of success, a negative error code otherwise */ -static int extract_header(AVCodecContext *const avctx, - const AVPacket *const avpkt) +static int extract_header(AVCodecContext *const avctx) { IffContext *s = avctx->priv_data; const uint8_t *buf; @@ -216,55 +214,6 @@ static int extract_header(AVCodecContext *const avctx, } palette_size = avctx->extradata_size - AV_RB16(avctx->extradata); - if (avpkt && avctx->codec_tag == MKTAG('A', 'N', 'I', 'M')) { - uint32_t chunk_id; - uint64_t data_size; - GetByteContext *gb = &s->gb; - - bytestream2_skip(gb, 4); - while (bytestream2_get_bytes_left(gb) >= 1) { - chunk_id = bytestream2_get_le32(gb); - data_size = bytestream2_get_be32(gb); - - if (chunk_id == MKTAG('B', 'M', 'H', 'D')) { - bytestream2_skip(gb, data_size + (data_size & 1)); - } else if (chunk_id == MKTAG('A', 'N', 'H', 'D')) { - unsigned extra; - if (data_size < 40) - return AVERROR_INVALIDDATA; - - s->compression = (bytestream2_get_byte(gb) << 8) | (s->compression & 0xFF); - bytestream2_skip(gb, 19); - extra = bytestream2_get_be32(gb); - s->is_short = !(extra & 1); - s->is_brush = extra == 2; - s->is_interlaced = !!(extra & 0x40); - data_size -= 24; - bytestream2_skip(gb, data_size + (data_size & 1)); - } else if (chunk_id == MKTAG('D', 'L', 'T', 'A') || - chunk_id == MKTAG('B', 'O', 'D', 'Y')) { - if (chunk_id == MKTAG('B','O','D','Y')) - s->compression &= 0xFF; - break; - } else if (chunk_id == MKTAG('C', 'M', 'A', 'P')) { - int count = data_size / 3; - uint32_t *pal = s->pal; - - if (count > 256) - return AVERROR_INVALIDDATA; - if (s->ham) { - for (i = 0; i < count; i++) - pal[i] = 0xFF000000 | bytestream2_get_le24(gb); - } else { - for (i = 0; i < count; i++) - pal[i] = 0xFF000000 | bytestream2_get_be24(gb); - } - bytestream2_skip(gb, data_size & 1); - } else { - bytestream2_skip(gb, data_size + (data_size&1)); - } - } - } else if (!avpkt) { buf = avctx->extradata; buf_size = bytestream_get_be16(&buf); if (buf_size <= 1 || palette_size < 0) { @@ -273,7 +222,6 @@ static int extract_header(AVCodecContext *const avctx, buf_size, palette_size); return AVERROR_INVALIDDATA; } - } if (buf_size >= 41) { s->compression = bytestream_get_byte(&buf); @@ -449,7 +397,7 @@ static av_cold int decode_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } - if ((err = extract_header(avctx, NULL)) < 0) + if ((err = extract_header(avctx)) < 0) return err; return 0; @@ -1525,6 +1473,64 @@ static int unsupported(AVCodecContext *avctx) return AVERROR_INVALIDDATA; } +static int parse_packet_header(AVCodecContext *const avctx, + GetByteContext *gb) +{ + IffContext *s = avctx->priv_data; + int i; + + if (avctx->codec_tag == MKTAG('A', 'N', 'I', 'M')) { + uint32_t chunk_id; + uint64_t data_size; + + bytestream2_skip(gb, 4); + while (bytestream2_get_bytes_left(gb) >= 1) { + chunk_id = bytestream2_get_le32(gb); + data_size = bytestream2_get_be32(gb); + + if (chunk_id == MKTAG('B', 'M', 'H', 'D')) { + bytestream2_skip(gb, data_size + (data_size & 1)); + } else if (chunk_id == MKTAG('A', 'N', 'H', 'D')) { + unsigned extra; + if (data_size < 40) + return AVERROR_INVALIDDATA; + + s->compression = (bytestream2_get_byte(gb) << 8) | (s->compression & 0xFF); + bytestream2_skip(gb, 19); + extra = bytestream2_get_be32(gb); + s->is_short = !(extra & 1); + s->is_brush = extra == 2; + s->is_interlaced = !!(extra & 0x40); + data_size -= 24; + bytestream2_skip(gb, data_size + (data_size & 1)); + } else if (chunk_id == MKTAG('D', 'L', 'T', 'A') || + chunk_id == MKTAG('B', 'O', 'D', 'Y')) { + if (chunk_id == MKTAG('B','O','D','Y')) + s->compression &= 0xFF; + break; + } else if (chunk_id == MKTAG('C', 'M', 'A', 'P')) { + int count = data_size / 3; + uint32_t *pal = s->pal; + + if (count > 256) + return AVERROR_INVALIDDATA; + if (s->ham) { + for (i = 0; i < count; i++) + pal[i] = 0xFF000000 | bytestream2_get_le24(gb); + } else { + for (i = 0; i < count; i++) + pal[i] = 0xFF000000 | bytestream2_get_be24(gb); + } + bytestream2_skip(gb, data_size & 1); + } else { + bytestream2_skip(gb, data_size + (data_size&1)); + } + } + } + + return 0; +} + static int decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) { @@ -1538,7 +1544,7 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *frame, bytestream2_init(gb, avpkt->data, avpkt->size); - if ((res = extract_header(avctx, avpkt)) < 0) + if ((res = parse_packet_header(avctx, gb)) < 0) return res; if ((res = ff_get_buffer(avctx, frame, 0)) < 0) -- 2.34.1

1 7

[PATCH 0/1] add motion_vec (motion vector) debug mode
by Chema Gonzalez 12 Sep '22

12 Sep '22

This is useful if you want to see the motion vectors (MV) generated by an encoder. I used ffmpeg debug mode, which is also used by similar functionality, like the QP value dumper. Chema Gonzalez (1): avcodec/mpegutils: add motion_vec debug mode libavcodec/avcodec.h | 1 + libavcodec/mpegutils.c | 45 +++++++++++++++++++++++++++++++++++++- libavcodec/options_table.h | 1 + 3 files changed, 46 insertions(+), 1 deletion(-) -- 2.37.2

4 14

[PATCH] avcodec/arm/sbcenc: save callee preserved vfp registers
by James Cowgill 12 Sep '22

12 Sep '22

When compiling FFmpeg with GCC-9, some very random segfaults were observed in code which had previously called down into the SBC encoder NEON assembly routines. This was caused by these functions clobbering some of the vfp callee saved registers (d8 - d15 aka q4 - q7). GCC was using these registers to save local variables, but after these functions returned, they would contain garbage. Fix by saving the relevant registers on the stack in the affected functions. Signed-off-by: James Cowgill <jcowgill(a)debian.org> --- libavcodec/arm/sbcdsp_neon.S | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavcodec/arm/sbcdsp_neon.S b/libavcodec/arm/sbcdsp_neon.S index d83d21d202..aa03800096 100644 --- a/libavcodec/arm/sbcdsp_neon.S +++ b/libavcodec/arm/sbcdsp_neon.S @@ -38,6 +38,8 @@ function ff_sbc_analyze_4_neon, export=1 /* TODO: merge even and odd cases (or even merge all four calls to this * function) in order to have only aligned reads from 'in' array * and reduce number of load instructions */ + vpush {d8-d11} + vld1.16 {d4, d5}, [r0, :64]! vld1.16 {d8, d9}, [r2, :128]! @@ -84,6 +86,7 @@ function ff_sbc_analyze_4_neon, export=1 vst1.32 {d0, d1}, [r1, :128] + vpop {d8-d11} bx lr endfunc @@ -91,6 +94,8 @@ function ff_sbc_analyze_8_neon, export=1 /* TODO: merge even and odd cases (or even merge all four calls to this * function) in order to have only aligned reads from 'in' array * and reduce number of load instructions */ + vpush {d8-d15} + vld1.16 {d4, d5}, [r0, :64]! vld1.16 {d8, d9}, [r2, :128]! @@ -188,6 +193,7 @@ function ff_sbc_analyze_8_neon, export=1 vst1.32 {d0, d1, d2, d3}, [r1, :128] + vpop {d8-d15} bx lr endfunc -- 2.22.0

3 3

[PATCH 1/1] lavf/dashdec: Multithreaded DASH initialization
by Lukas Fellechner 11 Sep '22

11 Sep '22

This patch adds multithreading support to DASH initialization. Initializing DASH streams is currently slow, because each individual stream is opened and probed sequentially. With DASH streams often having somewhere between 10-20 substreams, this can easily take up to half a minute on slow connections. This patch adds an "init-threads" option, specifying the max number of threads to use for parallel probing and initialization of substreams. If "init-threads" is set to a value larger than 1, multiple worker threads are spun up to massively bring down init times. Here is a free DASH stream for testing: http://www.bok.net/dash/tears_of_steel/cleartext/stream.mpd It has 7 substreams. I currently get init times of 3.5 seconds without patch. The patch brings it down to about 0.5 seconds, so using 7 threads nearly cut down init times by factor 7. On a slower connection (100mbit), the same stream took 7-8 seconds to initialize, the patch brought it down to just over 1 second. In the current patch, the behavior is disabled by default (init-threads = 0). But I think it could make sense to enable it by default, maybe with a reasonable value of 8? Not sure though, open for discussion. Some notes on the actual implementation: - DASH streams sometimes share a common init section. If this is the case, a mutex and condition is used, to make sure that the first stream reads the common section before the following streams start initialization. - Only the init and probing part is done in parallel. After all threads are joined, I collect the results and add the AVStreams to the parent AVFormatContext. That is why I split open_demux_for_component() into begin_open_demux_for_component() and end_open_demux_for_component(). - I tried to do this as clean as possible and added multiple comments. - Multithreading is never simple, so a proper review is needed. If this gets merged, I might try to do the same for HLS. This is my first PR by the way, so please be nice :) Lukas

3 25

[PATCH 1/3] avcodec/fmvc: Move frame allocation to a later stage
by Michael Niedermayer 08 Sep '22

08 Sep '22

This way more things are checked before allocation Signed-off-by: Michael Niedermayer <michael(a)niedermayer.cc> --- libavcodec/fmvc.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/libavcodec/fmvc.c b/libavcodec/fmvc.c index 4abf6d7048..912ad8fc82 100644 --- a/libavcodec/fmvc.c +++ b/libavcodec/fmvc.c @@ -401,20 +401,17 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *frame, GetByteContext *gb = &s->gb; PutByteContext *pb = &s->pb; int ret, y, x; + int key_frame; if (avpkt->size < 8) return AVERROR_INVALIDDATA; - if ((ret = ff_get_buffer(avctx, frame, 0)) < 0) - return ret; - bytestream2_init(gb, avpkt->data, avpkt->size); bytestream2_skip(gb, 2); - frame->key_frame = !!bytestream2_get_le16(gb); - frame->pict_type = frame->key_frame ? AV_PICTURE_TYPE_I : AV_PICTURE_TYPE_P; + key_frame = !!bytestream2_get_le16(gb); - if (frame->key_frame) { + if (key_frame) { const uint8_t *src; unsigned type, size; uint8_t *dst; @@ -434,6 +431,12 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *frame, return AVERROR_PATCHWELCOME; } + if ((ret = ff_get_buffer(avctx, frame, 0)) < 0) + return ret; + + frame->key_frame = 1; + frame->pict_type = AV_PICTURE_TYPE_I; + src = s->buffer; dst = frame->data[0] + (avctx->height - 1) * frame->linesize[0]; for (y = 0; y < avctx->height; y++) { @@ -514,6 +517,12 @@ static int decode_frame(AVCodecContext *avctx, AVFrame *frame, dst = &rect[block_h * s->stride]; } + if ((ret = ff_get_buffer(avctx, frame, 0)) < 0) + return ret; + + frame->key_frame = 0; + frame->pict_type = AV_PICTURE_TYPE_P; + ssrc = s->buffer; ddst = frame->data[0] + (avctx->height - 1) * frame->linesize[0]; for (y = 0; y < avctx->height; y++) { -- 2.17.1

3 15

[PATCH] avfilter/vf_showinfo: remove backspaces
by Michael Niedermayer 08 Sep '22

08 Sep '22

They mess with storing editing and comparing the results Signed-off-by: Michael Niedermayer <michael(a)niedermayer.cc> --- libavfilter/vf_showinfo.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_showinfo.c b/libavfilter/vf_showinfo.c index 2c8514fc80..9b4a9fc981 100644 --- a/libavfilter/vf_showinfo.c +++ b/libavfilter/vf_showinfo.c @@ -733,12 +733,15 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) av_log(ctx, AV_LOG_INFO, " %08"PRIX32, plane_checksum[plane]); av_log(ctx, AV_LOG_INFO, "] mean:["); for (plane = 0; plane < 4 && frame->data[plane] && frame->linesize[plane]; plane++) - av_log(ctx, AV_LOG_INFO, "%"PRId64" ", (sum[plane] + pixelcount[plane]/2) / pixelcount[plane]); - av_log(ctx, AV_LOG_INFO, "\b] stdev:["); + av_log(ctx, AV_LOG_INFO, "%s%"PRId64, + plane ? " ":"", + (sum[plane] + pixelcount[plane]/2) / pixelcount[plane]); + av_log(ctx, AV_LOG_INFO, "] stdev:["); for (plane = 0; plane < 4 && frame->data[plane] && frame->linesize[plane]; plane++) - av_log(ctx, AV_LOG_INFO, "%3.1f ", + av_log(ctx, AV_LOG_INFO, "%s%3.1f", + plane ? " ":"", sqrt((sum2[plane] - sum[plane]*(double)sum[plane]/pixelcount[plane])/pixelcount[plane])); - av_log(ctx, AV_LOG_INFO, "\b]"); + av_log(ctx, AV_LOG_INFO, "]"); } av_log(ctx, AV_LOG_INFO, "\n"); -- 2.17.1

1 1

[PATCH 1/2] avformat/imfdec: check track valid before use it
by Steven Liu 08 Sep '22

08 Sep '22

fix CID: 1512414 Signed-off-by: Steven Liu <lq(a)chinaffmpeg.org> --- libavformat/imfdec.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libavformat/imfdec.c b/libavformat/imfdec.c index 5bbe7a53f8..a97dcc3b8b 100644 --- a/libavformat/imfdec.c +++ b/libavformat/imfdec.c @@ -697,8 +697,9 @@ static IMFVirtualTrackPlaybackCtx *get_next_track_with_minimum_timestamp(AVForma } } - av_log(s, AV_LOG_DEBUG, "Found next track to read: %d (timestamp: %lf / %lf)\n", - track->index, av_q2d(track->current_timestamp), av_q2d(minimum_timestamp)); + if (track) + av_log(s, AV_LOG_DEBUG, "Found next track to read: %d (timestamp: %lf / %lf)\n", + track->index, av_q2d(track->current_timestamp), av_q2d(minimum_timestamp)); return track; } -- 2.25.0

4 11