[PATCH] av1_in_ts_v2 (PR #21307)
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch This patch series V2 adds support for carrying AV1 video streams in MPEG-2 Transport Stream containers, following the AOM "Carriage of AV1 in MPEG-2 TS" specification. Key Features - MPEG-TS Muxer: Converts AV1 from Section 5 (low overhead) format to start code format, where each OBU is prefixed with a 0x000001 start code. Includes Registration Descriptor ('AV01') and AV1 Video Descriptor (0x80) in PMT. - MPEG-TS Demuxer: Identifies AV1 streams by stream_type 0x06 (private data) and Registration Descriptor. Outputs AV1 data in start code format. - av1_tstosection5 BSF: Converts AV1 from MPEG-TS start code format back to Section 5 format for compatibility with other containers and decoders. Automatically inserted when remuxing to MP4, MKV, FLV, IVF, and raw OBU files. - Decoder Support: Both libdav1d and libaom decoders can directly decode AV1 in start code format. From 30f8ecd7c946e1a422318c547b4c663e8e27c53f Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:11:18 +0800 Subject: [PATCH 01/16] lavc/av1_parse: add start code format parsing support Add functions to parse AV1 OBUs in MPEG-TS start code format (0x000001 prefix). This is needed for AV1 streams from MPEG-TS containers per AOM AV1 in MPEG-2 TS specification. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavcodec/av1_parse.c | 133 +++++++++++++++++++++++++++++++++++++++++ libavcodec/av1_parse.h | 36 +++++++++++ 2 files changed, 169 insertions(+) diff --git a/libavcodec/av1_parse.c b/libavcodec/av1_parse.c index 061636815f..146c8aff79 100644 --- a/libavcodec/av1_parse.c +++ b/libavcodec/av1_parse.c @@ -20,6 +20,7 @@ #include "config.h" +#include "libavutil/intreadwrite.h" #include "libavutil/mem.h" #include "av1.h" @@ -120,3 +121,135 @@ AVRational ff_av1_framerate(int64_t ticks_per_frame, int64_t units_per_tick, return (AVRational){ 0, 1 }; } + +int ff_av1_is_startcode_format(const uint8_t *buf, int length) +{ + if (length < 4) + return 0; + + /* Check for 3-byte start code (MPEG-TS AV1 spec) */ + if (AV_RB24(buf) == 0x000001) + return 1; + + return 0; +} + +/** + * Find the next 3-byte start code (0x000001) in buffer. + * Note: MPEG-TS AV1 uses only 3-byte start codes per the specification. + */ +static const uint8_t *find_next_startcode(const uint8_t *p, const uint8_t *end) +{ + while (p + 3 <= end) { + if (AV_RB24(p) == 0x000001) + return p; + p++; + } + return end; +} + +int ff_av1_extract_obu_startcode(AV1OBU *obu, const uint8_t *buf, int length, + void *logctx) +{ + int start_code_size = 3; /* MPEG-TS AV1 uses only 3-byte start codes */ + int64_t obu_size; + int start_pos, type, temporal_id, spatial_id; + int ret; + + if (length < 4) + return AVERROR_INVALIDDATA; + + /* Verify 3-byte start code */ + if (AV_RB24(buf) != 0x000001) { + av_log(logctx, AV_LOG_ERROR, "Invalid AV1 start code\n"); + return AVERROR_INVALIDDATA; + } + + if (length - start_code_size < 1) + return AVERROR_INVALIDDATA; + + /* Parse OBU header after start code */ + ret = parse_obu_header(buf + start_code_size, + length - start_code_size, + &obu_size, &start_pos, &type, + &temporal_id, &spatial_id); + if (ret < 0) { + av_log(logctx, AV_LOG_ERROR, "Failed to parse OBU header\n"); + return ret; + } + + obu->type = type; + obu->temporal_id = temporal_id; + obu->spatial_id = spatial_id; + obu->raw_data = buf + start_code_size; + obu->raw_size = ret; + obu->data = buf + start_code_size + start_pos; + obu->size = obu_size; + obu->size_bits = get_obu_bit_length(obu->data, obu->size, type); + + av_log(logctx, AV_LOG_DEBUG, + "startcode obu_type: %d, temporal_id: %d, spatial_id: %d, payload size: %d\n", + obu->type, obu->temporal_id, obu->spatial_id, obu->size); + + return start_code_size + ret; +} + +int ff_av1_packet_split_startcode(AV1Packet *pkt, const uint8_t *buf, + int length, void *logctx) +{ + const uint8_t *p = buf; + const uint8_t *end = buf + length; + int ret; + + pkt->nb_obus = 0; + + /* Find first start code */ + p = find_next_startcode(p, end); + + while (p < end) { + AV1OBU *obu; + int consumed; + + /* Ensure OBU array has space */ + if (pkt->obus_allocated < pkt->nb_obus + 1) { + int new_size = pkt->obus_allocated + 1; + AV1OBU *tmp; + + if (new_size >= INT_MAX / sizeof(*tmp)) + return AVERROR(ENOMEM); + tmp = av_fast_realloc(pkt->obus, &pkt->obus_allocated_size, + new_size * sizeof(*tmp)); + if (!tmp) + return AVERROR(ENOMEM); + + pkt->obus = tmp; + memset(pkt->obus + pkt->obus_allocated, 0, sizeof(*pkt->obus)); + pkt->obus_allocated = new_size; + } + + obu = &pkt->obus[pkt->nb_obus]; + + /* Extract OBU using its size field, not by finding next start code */ + ret = ff_av1_extract_obu_startcode(obu, p, end - p, logctx); + if (ret < 0) { + av_log(logctx, AV_LOG_WARNING, + "Failed to extract OBU at offset %td, skipping to next start code\n", + p - buf); + /* On error, try to find next start code and continue */ + p = find_next_startcode(p + 3, end); + continue; + } + + consumed = ret; /* Total bytes consumed including start code */ + + pkt->nb_obus++; + p += consumed; + + /* Skip to next start code if there's gap */ + if (p < end && AV_RB24(p) != 0x000001) { + p = find_next_startcode(p, end); + } + } + + return 0; +} diff --git a/libavcodec/av1_parse.h b/libavcodec/av1_parse.h index 2b8cce4835..cfbef53a1b 100644 --- a/libavcodec/av1_parse.h +++ b/libavcodec/av1_parse.h @@ -169,4 +169,40 @@ static inline int get_obu_bit_length(const uint8_t *buf, int size, int type) AVRational ff_av1_framerate(int64_t ticks_per_frame, int64_t units_per_tick, int64_t time_scale); +/** + * Check if data is in MPEG-TS start code format. + * + * MPEG-TS AV1 uses start codes (0x000001) to delimit OBUs, + * unlike Section 5 (Low Overhead) format. + * + * @param buf input buffer + * @param length buffer length + * @return 1 if start code format, 0 otherwise + */ +int ff_av1_is_startcode_format(const uint8_t *buf, int length); + +/** + * Extract a single OBU from MPEG-TS start code format. + * + * @param obu output OBU structure + * @param buf input buffer (should start with 0x000001 or 0x00000001) + * @param length input length + * @param logctx logging context + * @return bytes consumed on success, negative error code on failure + */ +int ff_av1_extract_obu_startcode(AV1OBU *obu, const uint8_t *buf, int length, + void *logctx); + +/** + * Split start code format packet into OBU list. + * + * @param pkt output OBU list + * @param buf input buffer + * @param length input length + * @param logctx logging context + * @return 0 on success, negative error code on failure + */ +int ff_av1_packet_split_startcode(AV1Packet *pkt, const uint8_t *buf, + int length, void *logctx); + #endif /* AVCODEC_AV1_PARSE_H */ -- 2.49.1 From 46dbfa94d86d193dad1065f2f75ba6e2504458ef Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:11:35 +0800 Subject: [PATCH 02/16] lavc/av1_parser: add start code format detection and conversion Detect MPEG-TS start code format input and convert to Section 5 format for CBS parsing. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavcodec/av1_parser.c | 85 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 84 insertions(+), 1 deletion(-) diff --git a/libavcodec/av1_parser.c b/libavcodec/av1_parser.c index 32135a23cb..ba3710853b 100644 --- a/libavcodec/av1_parser.c +++ b/libavcodec/av1_parser.c @@ -22,6 +22,7 @@ #include "libavutil/attributes.h" #include "libavutil/avassert.h" +#include "libavutil/mem.h" #include "av1_parse.h" #include "avcodec.h" @@ -33,6 +34,9 @@ typedef struct AV1ParseContext { CodedBitstreamContext *cbc; CodedBitstreamFragment temporal_unit; int parsed_extradata; + + /* Start code format detection for MPEG-TS input */ + int in_startcode_mode; } AV1ParseContext; static const enum AVPixelFormat pix_fmts_8bit[2][2] = { @@ -52,6 +56,54 @@ static const enum AVPixelFormat pix_fmts_rgb[3] = { AV_PIX_FMT_GBRP, AV_PIX_FMT_GBRP10, AV_PIX_FMT_GBRP12, }; +/** + * Convert start code format to Section 5 format for parsing. + */ +static int convert_startcode_to_section5(const uint8_t *src, int src_size, + uint8_t **dst, int *dst_size, + void *logctx) +{ + AV1Packet pkt = { 0 }; + int ret, i; + size_t total_size = 0; + uint8_t *out, *p; + + ret = ff_av1_packet_split_startcode(&pkt, src, src_size, logctx); + if (ret < 0) + return ret; + + /* Calculate output size */ + for (i = 0; i < pkt.nb_obus; i++) + total_size += pkt.obus[i].raw_size; + + if (total_size == 0) { + ff_av1_packet_uninit(&pkt); + *dst = NULL; + *dst_size = 0; + return 0; + } + + out = av_malloc(total_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!out) { + ff_av1_packet_uninit(&pkt); + return AVERROR(ENOMEM); + } + + /* Copy OBUs without start codes */ + p = out; + for (i = 0; i < pkt.nb_obus; i++) { + memcpy(p, pkt.obus[i].raw_data, pkt.obus[i].raw_size); + p += pkt.obus[i].raw_size; + } + memset(p, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + ff_av1_packet_uninit(&pkt); + + *dst = out; + *dst_size = total_size; + return 0; +} + static int av1_parser_parse(AVCodecParserContext *ctx, AVCodecContext *avctx, const uint8_t **out_data, int *out_size, @@ -62,6 +114,10 @@ static int av1_parser_parse(AVCodecParserContext *ctx, const CodedBitstreamAV1Context *av1 = s->cbc->priv_data; const AV1RawSequenceHeader *seq; const AV1RawColorConfig *color; + uint8_t *converted_data = NULL; + int converted_size = 0; + const uint8_t *parse_data; + int parse_size; int ret; *out_data = data; @@ -71,6 +127,25 @@ static int av1_parser_parse(AVCodecParserContext *ctx, ctx->pict_type = AV_PICTURE_TYPE_NONE; ctx->picture_structure = AV_PICTURE_STRUCTURE_UNKNOWN; + /* Detect and handle start code format from MPEG-TS */ + if (ff_av1_is_startcode_format(data, size)) { + s->in_startcode_mode = 1; + av_log(avctx, AV_LOG_DEBUG, "Detected AV1 start code format input\n"); + + ret = convert_startcode_to_section5(data, size, &converted_data, + &converted_size, avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to convert start code format\n"); + return size; + } + + parse_data = converted_data; + parse_size = converted_size; + } else { + parse_data = data; + parse_size = size; + } + s->cbc->log_ctx = avctx; if (avctx->extradata_size && !s->parsed_extradata) { @@ -84,7 +159,14 @@ static int av1_parser_parse(AVCodecParserContext *ctx, ff_cbs_fragment_reset(td); } - ret = ff_cbs_read(s->cbc, td, NULL, data, size); + if (parse_size == 0) { + av_freep(&converted_data); + goto end_no_parse; + } + + ret = ff_cbs_read(s->cbc, td, NULL, parse_data, parse_size); + av_freep(&converted_data); + if (ret < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to parse temporal unit.\n"); goto end; @@ -174,6 +256,7 @@ static int av1_parser_parse(AVCodecParserContext *ctx, end: ff_cbs_fragment_reset(td); +end_no_parse: s->cbc->log_ctx = NULL; return size; -- 2.49.1 From 089eddfe29a87652b1812dce299871640ab45868 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:12:47 +0800 Subject: [PATCH 03/16] lavc/bsf: add av1_tstosection5 bitstream filter Convert AV1 from MPEG-TS start code format (0x000001 prefix) to Section 5 (low overhead) format. This is needed when remuxing AV1 streams from MPEG-TS to other containers like MP4, MKV, or IVF. Acts as no-op if input is already in Section 5 format, following the pattern of h264_mp4toannexb and hevc_mp4toannexb BSFs. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 1 + libavcodec/bsf/av1_tstosection5.c | 191 ++++++++++++++++++++++++++++++ 3 files changed, 193 insertions(+) create mode 100644 libavcodec/bsf/av1_tstosection5.c diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index b4b852c7e6..f32beec8fe 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -30,6 +30,7 @@ extern const FFBitStreamFilter ff_apv_metadata_bsf; extern const FFBitStreamFilter ff_av1_frame_merge_bsf; extern const FFBitStreamFilter ff_av1_frame_split_bsf; extern const FFBitStreamFilter ff_av1_metadata_bsf; +extern const FFBitStreamFilter ff_av1_tstosection5_bsf; extern const FFBitStreamFilter ff_chomp_bsf; extern const FFBitStreamFilter ff_dump_extradata_bsf; extern const FFBitStreamFilter ff_dca_core_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index 360fd1e32f..b1c9e459be 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -7,6 +7,7 @@ OBJS-$(CONFIG_APV_METADATA_BSF) += bsf/apv_metadata.o OBJS-$(CONFIG_AV1_FRAME_MERGE_BSF) += bsf/av1_frame_merge.o OBJS-$(CONFIG_AV1_FRAME_SPLIT_BSF) += bsf/av1_frame_split.o OBJS-$(CONFIG_AV1_METADATA_BSF) += bsf/av1_metadata.o +OBJS-$(CONFIG_AV1_TSTOSECTION5_BSF) += bsf/av1_tstosection5.o OBJS-$(CONFIG_CHOMP_BSF) += bsf/chomp.o OBJS-$(CONFIG_DCA_CORE_BSF) += bsf/dca_core.o OBJS-$(CONFIG_DTS2PTS_BSF) += bsf/dts2pts.o diff --git a/libavcodec/bsf/av1_tstosection5.c b/libavcodec/bsf/av1_tstosection5.c new file mode 100644 index 0000000000..2981602d70 --- /dev/null +++ b/libavcodec/bsf/av1_tstosection5.c @@ -0,0 +1,191 @@ +/* + * AV1 MPEG-TS to Section 5 (Low Overhead) bitstream filter + * Copyright (c) 2025 Jun Zhao <barryjzhao@tencent.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * This bitstream filter converts AV1 from MPEG-TS start code format + * to Section 5 (Low Overhead) format. + * + * If the input is already in Section 5 format, it passes through unchanged. + * + * Note: Per AOM AV1-MPEG2-TS spec section 3.6.2.1, emulation prevention bytes + * should be handled, but for now we rely on the obu_size field for boundary + * detection which makes emulation prevention optional in practice. + */ + +#include "libavutil/mem.h" + +#include "bsf.h" +#include "bsf_internal.h" +#include "av1.h" +#include "av1_parse.h" + +typedef struct AV1TsToSection5Context { + AVPacket *buffer_pkt; + + uint8_t *output_buffer; + size_t output_buffer_size; + size_t output_buffer_capacity; +} AV1TsToSection5Context; + +static int ensure_output_buffer(AV1TsToSection5Context *s, size_t required) +{ + if (s->output_buffer_capacity >= required) + return 0; + + size_t new_capacity = FFMAX(required, s->output_buffer_capacity * 2); + new_capacity = FFMAX(new_capacity, 4096); + + uint8_t *new_buffer = av_realloc(s->output_buffer, + new_capacity + AV_INPUT_BUFFER_PADDING_SIZE); + if (!new_buffer) + return AVERROR(ENOMEM); + + s->output_buffer = new_buffer; + s->output_buffer_capacity = new_capacity; + return 0; +} + +static int convert_startcode_to_section5(AV1TsToSection5Context *s, + const uint8_t *src, int src_size, + void *logctx) +{ + AV1Packet pkt = { 0 }; + int ret, i; + size_t total_size = 0; + uint8_t *p; + + /* Parse start code format */ + ret = ff_av1_packet_split_startcode(&pkt, src, src_size, logctx); + if (ret < 0) + return ret; + + /* Calculate output size (Section 5 format without start codes) */ + for (i = 0; i < pkt.nb_obus; i++) { + total_size += pkt.obus[i].raw_size; + } + + /* Ensure output buffer capacity */ + ret = ensure_output_buffer(s, total_size); + if (ret < 0) { + ff_av1_packet_uninit(&pkt); + return ret; + } + + /* Write Section 5 format (no start codes) */ + p = s->output_buffer; + for (i = 0; i < pkt.nb_obus; i++) { + memcpy(p, pkt.obus[i].raw_data, pkt.obus[i].raw_size); + p += pkt.obus[i].raw_size; + } + + s->output_buffer_size = total_size; + + /* Fill padding */ + memset(s->output_buffer + total_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + ff_av1_packet_uninit(&pkt); + return 0; +} + +static int av1_ts_to_section5_filter(AVBSFContext *ctx, AVPacket *pkt) +{ + AV1TsToSection5Context *s = ctx->priv_data; + int ret; + + ret = ff_bsf_get_packet_ref(ctx, s->buffer_pkt); + if (ret < 0) + return ret; + + /* If already Section 5 format, pass through (no-op) */ + if (!ff_av1_is_startcode_format(s->buffer_pkt->data, s->buffer_pkt->size)) { + av_packet_move_ref(pkt, s->buffer_pkt); + return 0; + } + + /* Convert format */ + ret = convert_startcode_to_section5(s, s->buffer_pkt->data, + s->buffer_pkt->size, ctx); + if (ret < 0) { + av_packet_unref(s->buffer_pkt); + return ret; + } + + /* Create output packet */ + ret = av_new_packet(pkt, s->output_buffer_size); + if (ret < 0) { + av_packet_unref(s->buffer_pkt); + return ret; + } + + memcpy(pkt->data, s->output_buffer, s->output_buffer_size); + + /* Copy metadata */ + ret = av_packet_copy_props(pkt, s->buffer_pkt); + if (ret < 0) { + av_packet_unref(pkt); + av_packet_unref(s->buffer_pkt); + return ret; + } + + av_packet_unref(s->buffer_pkt); + return 0; +} + +static int av1_ts_to_section5_init(AVBSFContext *ctx) +{ + AV1TsToSection5Context *s = ctx->priv_data; + + s->buffer_pkt = av_packet_alloc(); + if (!s->buffer_pkt) + return AVERROR(ENOMEM); + + return 0; +} + +static void av1_ts_to_section5_flush(AVBSFContext *ctx) +{ + AV1TsToSection5Context *s = ctx->priv_data; + av_packet_unref(s->buffer_pkt); +} + +static void av1_ts_to_section5_close(AVBSFContext *ctx) +{ + AV1TsToSection5Context *s = ctx->priv_data; + + av_packet_free(&s->buffer_pkt); + av_freep(&s->output_buffer); +} + +static const enum AVCodecID av1_ts_to_section5_codec_ids[] = { + AV_CODEC_ID_AV1, AV_CODEC_ID_NONE, +}; + +const FFBitStreamFilter ff_av1_tstosection5_bsf = { + .p.name = "av1_tstosection5", + .p.codec_ids = av1_ts_to_section5_codec_ids, + .priv_data_size = sizeof(AV1TsToSection5Context), + .init = av1_ts_to_section5_init, + .flush = av1_ts_to_section5_flush, + .close = av1_ts_to_section5_close, + .filter = av1_ts_to_section5_filter, +}; + -- 2.49.1 From 8ca6212b38b5f9242ff7100678b121e9114dce30 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:13:06 +0800 Subject: [PATCH 04/16] lavf/mpegts: add AV1 demuxing support Add AV1 stream type (0x46) and AV1 Video Descriptor (0x80) parsing per AOM AV1 in MPEG-2 TS specification. Extract sequence parameters (profile, level, tier, bitdepth, chroma) from descriptor. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavformat/mpegts.c | 34 ++++++++++++++++++++++++++++++++++ libavformat/mpegts.h | 5 +++++ 2 files changed, 39 insertions(+) diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c index 7c19abaf76..7ae52bbe22 100644 --- a/libavformat/mpegts.c +++ b/libavformat/mpegts.c @@ -817,6 +817,7 @@ static const StreamType ISO_types[] = { { STREAM_TYPE_VIDEO_HEVC, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, { STREAM_TYPE_VIDEO_JPEGXS, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_JPEGXS }, { STREAM_TYPE_VIDEO_VVC, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, + { STREAM_TYPE_VIDEO_AV1, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_AV1 }, { STREAM_TYPE_VIDEO_CAVS, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_CAVS }, { STREAM_TYPE_VIDEO_DIRAC, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_DIRAC }, { STREAM_TYPE_VIDEO_AVS2, AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_AVS2 }, @@ -874,6 +875,7 @@ static const StreamType REGD_types[] = { { MKTAG('E', 'A', 'C', '3'), AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_EAC3 }, { MKTAG('H', 'E', 'V', 'C'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_HEVC }, { MKTAG('V', 'V', 'C', ' '), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_VVC }, + { MKTAG('A', 'V', '0', '1'), AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_AV1 }, { MKTAG('K', 'L', 'V', 'A'), AVMEDIA_TYPE_DATA, AV_CODEC_ID_SMPTE_KLV }, { MKTAG('V', 'A', 'N', 'C'), AVMEDIA_TYPE_DATA, AV_CODEC_ID_SMPTE_2038 }, { MKTAG('I', 'D', '3', ' '), AVMEDIA_TYPE_DATA, AV_CODEC_ID_TIMED_ID3 }, @@ -2294,6 +2296,38 @@ int ff_parse_mpeg2_descriptor(AVFormatContext *fc, AVStream *st, int stream_type sti->need_parsing = 0; } break; + case AV1_VIDEO_DESCRIPTOR: + /* Parse AV1 video descriptor per AOM "Carriage of AV1 in MPEG-2 TS" Section 2.2 */ + if (st->codecpar->codec_id == AV_CODEC_ID_AV1 && desc_len >= 4) { + int marker_version, seq_profile, seq_level_idx_0; + int seq_tier_0, high_bitdepth, twelve_bit, monochrome; + int byte1, byte2; + + marker_version = get8(pp, desc_end); + /* marker should be 1, version should be 1 */ + if ((marker_version & 0x80) && (marker_version & 0x7F) == 1) { + byte1 = get8(pp, desc_end); + byte2 = get8(pp, desc_end); + (void)get8(pp, desc_end); /* byte3: hdr_wcg_idc, reserved, etc. */ + + seq_profile = (byte1 >> 5) & 0x07; + seq_level_idx_0 = byte1 & 0x1F; + seq_tier_0 = (byte2 >> 7) & 0x01; + high_bitdepth = (byte2 >> 6) & 0x01; + twelve_bit = (byte2 >> 5) & 0x01; + monochrome = (byte2 >> 4) & 0x01; + + /* Set profile and level */ + st->codecpar->profile = seq_profile; + st->codecpar->level = seq_level_idx_0; + + av_log(fc, AV_LOG_TRACE, "AV1 video descriptor: profile=%d, level=%d, " + "tier=%d, bitdepth=%d, mono=%d\n", + seq_profile, seq_level_idx_0, seq_tier_0, + high_bitdepth ? (twelve_bit ? 12 : 10) : 8, monochrome); + } + } + break; case DOVI_VIDEO_STREAM_DESCRIPTOR: { uint32_t buf; diff --git a/libavformat/mpegts.h b/libavformat/mpegts.h index 223962d18e..c2bd5ca6e6 100644 --- a/libavformat/mpegts.h +++ b/libavformat/mpegts.h @@ -146,6 +146,7 @@ #define STREAM_TYPE_VIDEO_HEVC 0x24 #define STREAM_TYPE_VIDEO_JPEGXS 0x32 #define STREAM_TYPE_VIDEO_VVC 0x33 +#define STREAM_TYPE_VIDEO_AV1 0x06 /* Per AOM AV1-MPEG2-TS draft spec */ #define STREAM_TYPE_VIDEO_CAVS 0x42 #define STREAM_TYPE_VIDEO_AVS2 0xd2 #define STREAM_TYPE_VIDEO_AVS3 0xd4 @@ -230,6 +231,10 @@ https://developer.apple.com/library/archive/documentation/AudioVideo/Concept... https://professional.dolby.com/siteassets/content-creation/dolby-vision-for-... */ #define DOVI_VIDEO_STREAM_DESCRIPTOR 0xb0 +/** AV1 video descriptor per AOM "Carriage of AV1 in MPEG-2 TS" Section 2.2 + https://aomediacodec.github.io/av1-mpeg2-ts/ */ +#define AV1_VIDEO_DESCRIPTOR 0x80 + #define DATA_COMPONENT_DESCRIPTOR 0xfd /* ARIB STD-B10 */ typedef struct MpegTSContext MpegTSContext; -- 2.49.1 From e5c105d0bd22e87b1c1aac3d60f90cba129c0078 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:13:35 +0800 Subject: [PATCH 05/16] lavf/mpegtsenc: add AV1 muxing support Add AV1 muxing support per AOM AV1 in MPEG-2 TS specification: - Stream type 0x46 and 'AV01' registration descriptor - Convert Section 5 format to start code format (0x000001 prefix) - Write AV1 Video Descriptor (0x80) with sequence parameters Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavformat/mpegtsenc.c | 221 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 221 insertions(+) diff --git a/libavformat/mpegtsenc.c b/libavformat/mpegtsenc.c index ea7c6065a0..ff7e2d131f 100644 --- a/libavformat/mpegtsenc.c +++ b/libavformat/mpegtsenc.c @@ -34,9 +34,12 @@ #include "libavcodec/h264.h" #include "libavcodec/hevc/hevc.h" #include "libavcodec/vvc.h" +#include "libavcodec/av1.h" +#include "libavcodec/av1_parse.h" #include "libavcodec/startcode.h" #include "avformat.h" +#include "av1.h" #include "avio_internal.h" #include "internal.h" #include "mpegts.h" @@ -351,6 +354,86 @@ static void put_registration_descriptor(uint8_t **q_ptr, uint32_t tag) *q_ptr = q; } +/** + * Write AV1 video descriptor per AOM AV1 in MPEG-2 TS spec Section 2.2. + * + * The descriptor uses the same data structure as AV1CodecConfigurationRecord + * from ISOBMFF with two reserved bits repurposed for HDR/WCG identification. + */ +static int put_av1_video_descriptor(AVFormatContext *s, uint8_t **q_ptr, + AVCodecParameters *par) +{ + AV1SequenceParameters seq; + uint8_t *q = *q_ptr; + int ret; + uint8_t high_bitdepth, twelve_bit; + uint8_t hdr_wcg_idc = 0; /* SDR by default */ + + if (!par->extradata || par->extradata_size < 4) { + av_log(s, AV_LOG_WARNING, "AV1 extradata not available for video descriptor\n"); + return 0; /* Skip descriptor if no extradata */ + } + + ret = ff_av1_parse_seq_header(&seq, par->extradata, par->extradata_size); + if (ret < 0) { + av_log(s, AV_LOG_WARNING, "Failed to parse AV1 sequence header\n"); + return 0; /* Skip descriptor on error */ + } + + /* Determine high_bitdepth and twelve_bit from bitdepth */ + if (seq.bitdepth == 12) { + high_bitdepth = 1; + twelve_bit = 1; + } else if (seq.bitdepth == 10) { + high_bitdepth = 1; + twelve_bit = 0; + } else { + high_bitdepth = 0; + twelve_bit = 0; + } + + /* Determine HDR/WCG indicator based on color info */ + /* hdr_wcg_idc: 0=SDR, 1=WCG only, 2=HDR and WCG, 3=no indication */ + if (seq.color_description_present_flag) { + /* Check for HDR transfer characteristics (PQ or HLG) */ + int is_hdr = (seq.transfer_characteristics == 16 || /* PQ (SMPTE ST 2084) */ + seq.transfer_characteristics == 18); /* HLG (ARIB STD-B67) */ + /* Check for WCG (BT.2020 primaries) */ + int is_wcg = (seq.color_primaries == 9); /* BT.2020 */ + + if (is_hdr && is_wcg) + hdr_wcg_idc = 2; + else if (is_wcg) + hdr_wcg_idc = 1; + /* else SDR (0) */ + } + + /* descriptor_tag = 0x80 (AV1 video descriptor) */ + *q++ = 0x80; + /* descriptor_length = 4 */ + *q++ = 4; + /* Byte 0: marker (1) | version (7) = 0x81 (marker=1, version=1) */ + *q++ = 0x81; + /* Byte 1: seq_profile (3) | seq_level_idx_0 (5) */ + *q++ = (seq.profile << 5) | (seq.level & 0x1F); + /* Byte 2: seq_tier_0 (1) | high_bitdepth (1) | twelve_bit (1) | monochrome (1) | + * chroma_subsampling_x (1) | chroma_subsampling_y (1) | chroma_sample_position (2) */ + *q++ = (seq.tier << 7) | + (high_bitdepth << 6) | + (twelve_bit << 5) | + (seq.monochrome << 4) | + (seq.chroma_subsampling_x << 3) | + (seq.chroma_subsampling_y << 2) | + (seq.chroma_sample_position & 0x03); + /* Byte 3: hdr_wcg_idc (2) | reserved_zeros (1) | initial_presentation_delay_present (1) | + * reserved_zeros (4) */ + /* We don't use initial_presentation_delay, so set to 0 */ + *q++ = (hdr_wcg_idc << 6); + + *q_ptr = q; + return 0; +} + static int get_dvb_stream_type(AVFormatContext *s, AVStream *st) { MpegTSWrite *ts = s->priv_data; @@ -374,6 +457,9 @@ static int get_dvb_stream_type(AVFormatContext *s, AVStream *st) case AV_CODEC_ID_VVC: stream_type = STREAM_TYPE_VIDEO_VVC; break; + case AV_CODEC_ID_AV1: + stream_type = STREAM_TYPE_PRIVATE_DATA; + break; case AV_CODEC_ID_CAVS: stream_type = STREAM_TYPE_VIDEO_CAVS; break; @@ -807,6 +893,9 @@ static int mpegts_write_pmt(AVFormatContext *s, MpegTSService *service) put_registration_descriptor(&q, MKTAG('V', 'C', '-', '1')); } else if (stream_type == STREAM_TYPE_VIDEO_HEVC && s->strict_std_compliance <= FF_COMPLIANCE_NORMAL) { put_registration_descriptor(&q, MKTAG('H', 'E', 'V', 'C')); + } else if (codec_id == AV_CODEC_ID_AV1 && s->strict_std_compliance <= FF_COMPLIANCE_NORMAL) { + put_registration_descriptor(&q, MKTAG('A', 'V', '0', '1')); + put_av1_video_descriptor(s, &q, st->codecpar); } else if (stream_type == STREAM_TYPE_VIDEO_CAVS || stream_type == STREAM_TYPE_VIDEO_AVS2 || stream_type == STREAM_TYPE_VIDEO_AVS3) { put_registration_descriptor(&q, MKTAG('A', 'V', 'S', 'V')); @@ -1453,6 +1542,9 @@ static int get_pes_stream_id(AVFormatContext *s, AVStream *st, int stream_id, in if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) { if (st->codecpar->codec_id == AV_CODEC_ID_DIRAC) return STREAM_ID_EXTENDED_STREAM_ID; + else if (st->codecpar->codec_id == AV_CODEC_ID_AV1) + /* Per AOM AV1-MPEG2-TS draft spec: AV1 uses private_stream_1 (0xBD) */ + return STREAM_ID_PRIVATE_STREAM_1; else return STREAM_ID_VIDEO_STREAM_0; } else if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && @@ -1789,6 +1881,106 @@ int ff_check_h264_startcode(AVFormatContext *s, const AVStream *st, const AVPack return check_h26x_startcode(s, st, pkt, "h264"); } +/** + * Check if this is the start of a Temporal Unit. + */ +static int av1_is_temporal_unit_start(const uint8_t *data, int size) +{ + AV1Packet pkt = { 0 }; + int ret, result = 0; + + ret = ff_av1_packet_split(&pkt, data, size, NULL); + if (ret < 0) + return 1; /* Assume TU start on error */ + + /* Check if first OBU is TD */ + if (pkt.nb_obus > 0 && + pkt.obus[0].type == AV1_OBU_TEMPORAL_DELIMITER) { + result = 1; + } + + ff_av1_packet_uninit(&pkt); + return result; +} + +/** + * Convert Section 5 format to MPEG-TS start code format. + * + * @param is_tu_start whether this is the start of a Temporal Unit, + * determines if TD OBU should be inserted + * + * Note: Per AOM AV1-MPEG2-TS spec section 3.6.2.1, emulation prevention bytes + * should be handled, but for now we rely on the obu_size field for boundary + * detection which makes emulation prevention optional in practice. + */ +static int av1_section5_to_startcode(const uint8_t *src, int src_size, + uint8_t **dst, int *dst_size, + int is_tu_start, void *logctx) +{ + AV1Packet pkt = { 0 }; + int ret, i; + int total_size = 0; + uint8_t *out, *p; + + /* Parse Section 5 format OBUs */ + ret = ff_av1_packet_split(&pkt, src, src_size, logctx); + if (ret < 0) + return ret; + + /* Calculate output size */ + /* TD OBU if needed: 3 (start code) + 2 (header + size) */ + if (is_tu_start) + total_size += 5; + + for (i = 0; i < pkt.nb_obus; i++) { + /* Skip TD OBU in input (we control insertion) */ + if (pkt.obus[i].type == AV1_OBU_TEMPORAL_DELIMITER) + continue; + /* Each OBU: 3 (start code) + raw_size */ + total_size += 3 + pkt.obus[i].raw_size; + } + + out = av_malloc(total_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!out) { + ff_av1_packet_uninit(&pkt); + return AVERROR(ENOMEM); + } + p = out; + + /* Insert TD OBU at TU start */ + if (is_tu_start) { + /* Start code */ + AV_WB24(p, 0x000001); + p += 3; + /* TD OBU header: type=2, extension_flag=0, has_size_field=1 */ + *p++ = (AV1_OBU_TEMPORAL_DELIMITER << 3) | 0x02; + /* obu_size = 0 */ + *p++ = 0x00; + } + + /* Write other OBUs with start codes */ + for (i = 0; i < pkt.nb_obus; i++) { + if (pkt.obus[i].type == AV1_OBU_TEMPORAL_DELIMITER) + continue; + + /* Start code */ + AV_WB24(p, 0x000001); + p += 3; + /* OBU data */ + memcpy(p, pkt.obus[i].raw_data, pkt.obus[i].raw_size); + p += pkt.obus[i].raw_size; + } + + memset(p, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + ff_av1_packet_uninit(&pkt); + + *dst = out; + *dst_size = total_size; + + return 0; +} + /* Based on GStreamer's gst-plugins-base/ext/ogg/gstoggstream.c * Released under the LGPL v2.1+, written by * Vincent Penquerc'h <vincent.penquerch@collabora.co.uk> @@ -2063,6 +2255,22 @@ static int mpegts_write_packet_internal(AVFormatContext *s, AVPacket *pkt) if (!data) return AVERROR(ENOMEM); } + } else if (st->codecpar->codec_id == AV_CODEC_ID_AV1) { + int is_tu_start; + int new_size = 0; + int av1_ret; + + /* Check if this is start of Temporal Unit */ + is_tu_start = av1_is_temporal_unit_start(buf, size); + + /* Convert Section 5 to start code format */ + av1_ret = av1_section5_to_startcode(buf, size, &data, &new_size, + is_tu_start, s); + if (av1_ret < 0) + return av1_ret; + + buf = data; + size = new_size; } else if (st->codecpar->codec_id == AV_CODEC_ID_OPUS) { if (pkt->size < 2) { av_log(s, AV_LOG_ERROR, "Opus packet too short\n"); @@ -2325,6 +2533,19 @@ static int mpegts_check_bitstream(AVFormatContext *s, AVStream *st, ((st->codecpar->extradata[0] & e->mask) == e->value)))) return ff_stream_add_bitstream_filter(st, e->bsf_name, NULL); } + + /* AV1: Per MPEG-TS AV1 spec section 3.3, each PES packet should contain a + * single Access Unit (frame's OBUs). The av1_frame_split BSF can split + * multi-frame Temporal Units, but this causes DTS conflicts because all + * split frames inherit the same DTS, and modifying DTS causes pts < dts + * errors when the shown frame is not first in decode order. + * + * Most AV1 encoders (libaom, SVT-AV1) produce single-frame TUs, so the + * current implementation works without av1_frame_split. For multi-frame TUs + * (e.g., SVC content), users should manually apply av1_frame_split with + * appropriate timestamp handling. + */ + return 1; } -- 2.49.1 From 95b193dc201614613988eca0006c8587fc23007e Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:13:48 +0800 Subject: [PATCH 06/16] lavf/movenc: add automatic av1_tstosection5 BSF insertion Automatically insert av1_tstosection5 BSF when muxing AV1 streams from MPEG-TS start code format to MP4/MOV. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavformat/movenc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 8d8acd2aff..8de7308434 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -45,6 +45,7 @@ #include "libavcodec/internal.h" #include "libavcodec/put_bits.h" +#include "libavcodec/av1_parse.h" #include "libavcodec/vc1_common.h" #include "libavcodec/raw.h" #include "internal.h" @@ -8799,6 +8800,10 @@ static int mov_check_bitstream(AVFormatContext *s, AVStream *st, ret = ff_stream_add_bitstream_filter(st, "aac_adtstoasc", NULL); } else if (st->codecpar->codec_id == AV_CODEC_ID_VP9) { ret = ff_stream_add_bitstream_filter(st, "vp9_superframe", NULL); + } else if (st->codecpar->codec_id == AV_CODEC_ID_AV1) { + /* Convert MPEG-TS start code format to Section 5 if needed */ + if (ff_av1_is_startcode_format(pkt->data, pkt->size)) + ret = ff_stream_add_bitstream_filter(st, "av1_tstosection5", NULL); } return ret; -- 2.49.1 From 7e6fab1faee9819b953edcf6314754a5b6593c26 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:13:56 +0800 Subject: [PATCH 07/16] lavf/flvenc: add automatic av1_tstosection5 BSF insertion Automatically insert av1_tstosection5 BSF when muxing AV1 streams from MPEG-TS start code format to FLV. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavformat/flvenc.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavformat/flvenc.c b/libavformat/flvenc.c index a0503c1799..497e93f460 100644 --- a/libavformat/flvenc.c +++ b/libavformat/flvenc.c @@ -39,6 +39,7 @@ #include "nal.h" #include "mux.h" #include "libavutil/opt.h" +#include "libavcodec/av1_parse.h" #include "libavcodec/put_bits.h" @@ -1473,6 +1474,11 @@ static int flv_check_bitstream(AVFormatContext *s, AVStream *st, if (pkt->size > 2 && (AV_RB16(pkt->data) & 0xfff0) == 0xfff0) return ff_stream_add_bitstream_filter(st, "aac_adtstoasc", NULL); } + /* Convert MPEG-TS start code format to Section 5 if needed */ + if (st->codecpar->codec_id == AV_CODEC_ID_AV1 && + ff_av1_is_startcode_format(pkt->data, pkt->size)) + return ff_stream_add_bitstream_filter(st, "av1_tstosection5", NULL); + /* Extract extradata if needed */ if (!st->codecpar->extradata_size && (st->codecpar->codec_id == AV_CODEC_ID_H264 || st->codecpar->codec_id == AV_CODEC_ID_HEVC || -- 2.49.1 From 15bea4efcfa04c1ceb80664d22c62067dd87ccda Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:14:06 +0800 Subject: [PATCH 08/16] lavf/matroskaenc: add automatic av1_tstosection5 BSF insertion Automatically insert av1_tstosection5 BSF when muxing AV1 streams from MPEG-TS start code format to MKV/WebM. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavformat/matroskaenc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 18f17f4329..45f870e9f2 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -24,6 +24,7 @@ #include "config_components.h" #include "av1.h" +#include "libavcodec/av1_parse.h" #include "avc.h" #include "hevc.h" #include "avformat.h" @@ -3513,6 +3514,10 @@ static int mkv_check_bitstream(AVFormatContext *s, AVStream *st, } else if (CONFIG_MATROSKA_MUXER && st->codecpar->codec_id == AV_CODEC_ID_HDMV_PGS_SUBTITLE) { ret = ff_stream_add_bitstream_filter(st, "pgs_frame_merge", NULL); + } else if (st->codecpar->codec_id == AV_CODEC_ID_AV1) { + /* Convert MPEG-TS start code format to Section 5 if needed */ + if (ff_av1_is_startcode_format(pkt->data, pkt->size)) + return ff_stream_add_bitstream_filter(st, "av1_tstosection5", NULL); } return ret; -- 2.49.1 From 3193e5f52772e27f46942fc929b22c72b4efce7a Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:14:17 +0800 Subject: [PATCH 09/16] lavf/ivfenc: add automatic av1_tstosection5 BSF insertion Automatically insert av1_tstosection5 BSF when muxing AV1 streams from MPEG-TS start code format to IVF. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavformat/ivfenc.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/libavformat/ivfenc.c b/libavformat/ivfenc.c index 9feaea3516..267574fb09 100644 --- a/libavformat/ivfenc.c +++ b/libavformat/ivfenc.c @@ -21,6 +21,7 @@ #include "internal.h" #include "mux.h" #include "libavutil/intreadwrite.h" +#include "libavcodec/av1_parse.h" typedef struct IVFEncContext { unsigned frame_cnt; @@ -42,15 +43,26 @@ static int ivf_init(AVFormatContext *s) int ret = ff_stream_add_bitstream_filter(s->streams[0], "vp9_superframe", NULL); if (ret < 0) return ret; - } else if (par->codec_id == AV_CODEC_ID_AV1) { - int ret = ff_stream_add_bitstream_filter(s->streams[0], "av1_metadata", "td=insert"); - if (ret < 0) - return ret; } + /* AV1 BSFs (av1_tstosection5 and av1_metadata) are added in check_bitstream + * to properly handle MPEG-TS start code format input */ return 0; } +static int ivf_check_bitstream(AVFormatContext *s, AVStream *st, + const AVPacket *pkt) +{ + if (st->codecpar->codec_id == AV_CODEC_ID_AV1) { + /* Convert from MPEG-TS start code format to Section 5 if needed */ + if (ff_av1_is_startcode_format(pkt->data, pkt->size)) + return ff_stream_add_bitstream_filter(st, "av1_tstosection5", NULL); + /* Add av1_metadata to insert Temporal Delimiter OBUs */ + return ff_stream_add_bitstream_filter(st, "av1_metadata", "td=insert"); + } + return 1; +} + static int ivf_write_header(AVFormatContext *s) { AVCodecParameters *par = s->streams[0]->codecpar; @@ -125,4 +137,5 @@ const FFOutputFormat ff_ivf_muxer = { .write_header = ivf_write_header, .write_packet = ivf_write_packet, .write_trailer = ivf_write_trailer, + .check_bitstream = ivf_check_bitstream, }; -- 2.49.1 From 8032da04596c1773f738daf6e61d1b4033cbf2c6 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:14:25 +0800 Subject: [PATCH 10/16] lavf/rawenc: add automatic av1_tstosection5 BSF insertion for OBU Automatically insert av1_tstosection5 BSF when muxing AV1 streams from MPEG-TS start code format to OBU raw format. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavformat/rawenc.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index cf298d223d..4e03e24f9b 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -24,6 +24,8 @@ #include "libavutil/intreadwrite.h" +#include "libavcodec/av1_parse.h" + #include "avformat.h" #include "rawenc.h" #include "mux.h" @@ -569,6 +571,10 @@ const FFOutputFormat ff_mpeg2video_muxer = { static int obu_check_bitstream(AVFormatContext *s, AVStream *st, const AVPacket *pkt) { + /* Convert from MPEG-TS start code format to Section 5 if needed */ + if (ff_av1_is_startcode_format(pkt->data, pkt->size)) + return ff_stream_add_bitstream_filter(st, "av1_tstosection5", NULL); + return ff_stream_add_bitstream_filter(st, "av1_metadata", "td=insert"); } -- 2.49.1 From 14989d57511e54b36fbadb39be1c2f204a70040c Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:14:55 +0800 Subject: [PATCH 11/16] lavc/libdav1d: add MPEG-TS start code format support Automatically detect and convert MPEG-TS start code format to Section 5 format before passing to libdav1d. This allows direct decoding of AV1 streams from MPEG-TS without manual BSF application. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavcodec/libdav1d.c | 66 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/libavcodec/libdav1d.c b/libavcodec/libdav1d.c index 7f63bb1b15..67f30e52ba 100644 --- a/libavcodec/libdav1d.c +++ b/libavcodec/libdav1d.c @@ -299,6 +299,48 @@ static void libdav1d_user_data_free(const uint8_t *data, void *opaque) { av_packet_free(&pkt); } +/** + * Convert start code format to Section 5 format for libdav1d. + * This is needed when receiving AV1 data from MPEG-TS demuxer. + */ +static int libdav1d_convert_startcode(AVCodecContext *c, AVPacket *pkt) +{ + AV1Packet av1_pkt = { 0 }; + uint8_t *new_data; + size_t new_size = 0; + int ret, i; + + ret = ff_av1_packet_split_startcode(&av1_pkt, pkt->data, pkt->size, c); + if (ret < 0) + return ret; + + /* Calculate output size */ + for (i = 0; i < av1_pkt.nb_obus; i++) + new_size += av1_pkt.obus[i].raw_size; + + if (new_size == 0) { + ff_av1_packet_uninit(&av1_pkt); + return 0; + } + + /* Allocate new buffer */ + ret = av_new_packet(pkt, new_size); + if (ret < 0) { + ff_av1_packet_uninit(&av1_pkt); + return ret; + } + + /* Copy OBUs without start codes */ + new_data = pkt->data; + for (i = 0; i < av1_pkt.nb_obus; i++) { + memcpy(new_data, av1_pkt.obus[i].raw_data, av1_pkt.obus[i].raw_size); + new_data += av1_pkt.obus[i].raw_size; + } + + ff_av1_packet_uninit(&av1_pkt); + return 0; +} + static int libdav1d_receive_frame_internal(AVCodecContext *c, Dav1dPicture *p) { Libdav1dContext *dav1d = c->priv_data; @@ -318,6 +360,30 @@ static int libdav1d_receive_frame_internal(AVCodecContext *c, Dav1dPicture *p) } if (pkt->size) { + /* Convert MPEG-TS start code format to Section 5 if needed */ + if (ff_av1_is_startcode_format(pkt->data, pkt->size)) { + AVPacket *new_pkt = av_packet_alloc(); + if (!new_pkt) { + av_packet_free(&pkt); + return AVERROR(ENOMEM); + } + + res = av_packet_ref(new_pkt, pkt); + if (res < 0) { + av_packet_free(&pkt); + av_packet_free(&new_pkt); + return res; + } + av_packet_free(&pkt); + pkt = new_pkt; + + res = libdav1d_convert_startcode(c, pkt); + if (res < 0) { + av_packet_free(&pkt); + return res; + } + } + res = dav1d_data_wrap(data, pkt->data, pkt->size, libdav1d_data_free, pkt->buf); if (res < 0) { -- 2.49.1 From efc3c68c602f77471c4f92c2c261bf015801fd9a Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:15:04 +0800 Subject: [PATCH 12/16] lavc/libaomdec: add MPEG-TS start code format support Automatically detect and convert MPEG-TS start code format to Section 5 format before passing to libaom. This allows direct decoding of AV1 streams from MPEG-TS without manual BSF application. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- libavcodec/libaomdec.c | 71 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 1 deletion(-) diff --git a/libavcodec/libaomdec.c b/libavcodec/libaomdec.c index 9e9c4d18c5..cbd28626a5 100644 --- a/libavcodec/libaomdec.c +++ b/libavcodec/libaomdec.c @@ -30,8 +30,10 @@ #include "libavutil/cpu.h" #include "libavutil/hdr_dynamic_metadata.h" #include "libavutil/imgutils.h" +#include "libavutil/mem.h" #include "avcodec.h" +#include "av1_parse.h" #include "bytestream.h" #include "codec_internal.h" #include "decode.h" @@ -195,15 +197,79 @@ static int decode_metadata(AVFrame *frame, const struct aom_image *img) return 0; } +/** + * Convert start code format to Section 5 format for libaom. + * This is needed when receiving AV1 data from MPEG-TS demuxer. + */ +static int libaom_convert_startcode(AVCodecContext *avctx, + const uint8_t *src, int src_size, + uint8_t **dst, int *dst_size) +{ + AV1Packet pkt = { 0 }; + uint8_t *out; + size_t total_size = 0; + int ret, i; + + ret = ff_av1_packet_split_startcode(&pkt, src, src_size, avctx); + if (ret < 0) + return ret; + + /* Calculate output size */ + for (i = 0; i < pkt.nb_obus; i++) + total_size += pkt.obus[i].raw_size; + + if (total_size == 0) { + ff_av1_packet_uninit(&pkt); + *dst = NULL; + *dst_size = 0; + return 0; + } + + /* Allocate output buffer */ + out = av_malloc(total_size + AV_INPUT_BUFFER_PADDING_SIZE); + if (!out) { + ff_av1_packet_uninit(&pkt); + return AVERROR(ENOMEM); + } + + /* Copy OBUs without start codes */ + *dst_size = 0; + for (i = 0; i < pkt.nb_obus; i++) { + memcpy(out + *dst_size, pkt.obus[i].raw_data, pkt.obus[i].raw_size); + *dst_size += pkt.obus[i].raw_size; + } + memset(out + *dst_size, 0, AV_INPUT_BUFFER_PADDING_SIZE); + + ff_av1_packet_uninit(&pkt); + *dst = out; + return 0; +} + static int aom_decode(AVCodecContext *avctx, AVFrame *picture, int *got_frame, AVPacket *avpkt) { AV1DecodeContext *ctx = avctx->priv_data; const void *iter = NULL; struct aom_image *img; + const uint8_t *data = avpkt->data; + int size = avpkt->size; + uint8_t *converted_data = NULL; int ret; - if (aom_codec_decode(&ctx->decoder, avpkt->data, avpkt->size, NULL) != + /* Convert MPEG-TS start code format to Section 5 if needed */ + if (ff_av1_is_startcode_format(avpkt->data, avpkt->size)) { + int converted_size; + ret = libaom_convert_startcode(avctx, avpkt->data, avpkt->size, + &converted_data, &converted_size); + if (ret < 0) + return ret; + if (converted_data) { + data = converted_data; + size = converted_size; + } + } + + if (aom_codec_decode(&ctx->decoder, data, size, NULL) != AOM_CODEC_OK) { const char *error = aom_codec_error(&ctx->decoder); const char *detail = aom_codec_error_detail(&ctx->decoder); @@ -212,9 +278,12 @@ static int aom_decode(AVCodecContext *avctx, AVFrame *picture, if (detail) av_log(avctx, AV_LOG_ERROR, " Additional information: %s\n", detail); + av_freep(&converted_data); return AVERROR_INVALIDDATA; } + av_freep(&converted_data); + if ((img = aom_codec_get_frame(&ctx->decoder, &iter))) { if (img->d_w > img->w || img->d_h > img->h) { av_log(avctx, AV_LOG_ERROR, "Display dimensions %dx%d exceed storage %dx%d\n", -- 2.49.1 From 8597bc3e63d3b7355560da4a603c7aaf8d792639 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:15:43 +0800 Subject: [PATCH 13/16] doc/bitstream_filters: add av1_tstosection5 BSF documentation Document the av1_tstosection5 bitstream filter for converting AV1 from MPEG-TS start code format to Section 5 format. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- doc/bitstream_filters.texi | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index fb31ca7380..03a8abfc5d 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -92,6 +92,38 @@ Deletes Padding OBUs. @end table +@section av1_tstosection5 + +Convert AV1 bitstream from MPEG-TS start code format to Section 5 (low overhead) +format. + +This filter is used to convert AV1 streams from the format used in MPEG-TS +containers (with 0x000001 start codes prefixing each OBU) to the standard +Section 5 format used by most other containers and decoders. + +If the input is already in Section 5 format, this filter acts as a no-op and +passes the data through unchanged. + +This filter is automatically inserted when remuxing from MPEG-TS to containers +that require Section 5 format, such as MP4, MKV, FLV, IVF, and raw OBU files. + +@subsection Examples + +@itemize +@item +Convert an AV1 stream from MPEG-TS to MP4: +@example +ffmpeg -i input.ts -c:v copy output.mp4 +@end example +The filter is automatically inserted when needed. + +@item +Explicitly apply the filter: +@example +ffmpeg -i input.ts -bsf:v av1_tstosection5 -c:v copy output.ivf +@end example +@end itemize + @section chomp Remove zero padding at the end of a packet. -- 2.49.1 From 8e7edb18d1112cf00e1a61794a968235f6baa9f1 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:16:01 +0800 Subject: [PATCH 14/16] doc/muxers: add AV1 support documentation for mpegts muxer Document AV1 muxing support in MPEG-TS per AOM specification, including stream type, registration descriptor, and video descriptor. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- doc/muxers.texi | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/doc/muxers.texi b/doc/muxers.texi index a2e356187a..9f17bc5c42 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -3191,6 +3191,36 @@ ffmpeg -i file.mpg -c copy \ out.ts @end example +@subsection AV1 Support + +The MPEG-TS muxer supports AV1 video streams according to the AOM +"Carriage of AV1 in MPEG-2 TS" specification. + +AV1 streams are identified by stream type 0x06 (private data) and include: +@itemize +@item Registration descriptor with format identifier @samp{AV01} +@item AV1 video descriptor (descriptor tag 0x80) containing sequence parameters +@end itemize + +The muxer automatically converts AV1 from Section 5 (low overhead) format to +the start code format required by MPEG-TS, where each OBU is prefixed with +a 0x000001 start code. + +Temporal Delimiter OBUs are inserted at the beginning of each Temporal Unit +to mark the access unit boundaries. + +@subsubsection Example + +Mux an AV1 video file to MPEG-TS: +@example +ffmpeg -i input.mp4 -c:v copy -f mpegts output.ts +@end example + +Encode and mux AV1 to MPEG-TS: +@example +ffmpeg -i input.mp4 -c:v libaom-av1 -crf 30 -f mpegts output.ts +@end example + @section mxf, mxf_d10, mxf_opatom MXF muxer. -- 2.49.1 From 3a227dd7f19ba02a9c987e487b420edd07f110f4 Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:16:09 +0800 Subject: [PATCH 15/16] doc/demuxers: add AV1 support documentation for mpegts demuxer Document AV1 demuxing support in MPEG-TS, including stream type identification and av1_tstosection5 BSF usage for format conversion. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- doc/demuxers.texi | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/doc/demuxers.texi b/doc/demuxers.texi index c1dda7f1eb..93c842261d 100644 --- a/doc/demuxers.texi +++ b/doc/demuxers.texi @@ -1003,6 +1003,37 @@ Set maximum size, in bytes, of packet emitted by the demuxer. Payloads above thi are split across multiple packets. Range is 1 to INT_MAX/2. Default is 204800 bytes. @end table +@subsection AV1 Support + +The MPEG-TS demuxer supports AV1 video streams according to the AOM +"Carriage of AV1 in MPEG-2 TS" specification. + +AV1 streams are identified by: +@itemize +@item Stream type 0x06 (private data) in the PMT +@item Registration descriptor with format identifier @samp{AV01} +@end itemize + +The demuxer outputs AV1 data in the MPEG-TS start code format, where each OBU +is prefixed with a 0x000001 start code. To convert to the standard Section 5 +format used by other containers and decoders, the @code{av1_tstosection5} +bitstream filter can be used. + +When remuxing to other containers (MP4, MKV, FLV, IVF, etc.), the +@code{av1_tstosection5} filter is automatically inserted. + +The AV1 video descriptor (descriptor tag 0x80) is parsed to extract stream +parameters such as profile, level, tier, and color information. + +@subsubsection Example + +Demux and convert AV1 from MPEG-TS to MP4: +@example +ffmpeg -i input.ts -c:v copy output.mp4 +@end example + +The bitstream filter is automatically applied when needed. + @section mpjpeg MJPEG encapsulated in multi-part MIME demuxer. -- 2.49.1 From 33e1ce44bb62d0e4b8406b9ded8438376a56b51d Mon Sep 17 00:00:00 2001 From: Jun Zhao <barryjzhao@tencent.com> Date: Mon, 29 Dec 2025 01:16:23 +0800 Subject: [PATCH 16/16] fate: add AV1 in MPEG-TS tests Add FATE tests for AV1 muxing, demuxing, roundtrip, and BSF conversion in MPEG-TS. Signed-off-by: Jun Zhao <barryjzhao@tencent.com> --- tests/fate/cbs.mak | 10 ++++++++++ tests/fate/mpegts.mak | 17 ++++++++++++++++- tests/ref/fate/av1-bsf-tstosection5 | 11 +++++++++++ tests/ref/fate/mpegts-av1-mux | 10 ++++++++++ tests/ref/fate/mpegts-av1-roundtrip | 11 +++++++++++ 5 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 tests/ref/fate/av1-bsf-tstosection5 create mode 100644 tests/ref/fate/mpegts-av1-mux create mode 100644 tests/ref/fate/mpegts-av1-roundtrip diff --git a/tests/fate/cbs.mak b/tests/fate/cbs.mak index d9200debc9..eda673426f 100644 --- a/tests/fate/cbs.mak +++ b/tests/fate/cbs.mak @@ -66,6 +66,16 @@ FATE_CBS_AV1-$(call FATE_CBS_DEPS, IVF, AV1, AV1, AV1, RAWVIDEO) = $(FATE_CBS_av FATE_SAMPLES_AVCONV += $(FATE_CBS_AV1-yes) fate-cbs-av1: $(FATE_CBS_AV1-yes) +# AV1 MPEG-TS to Section 5 BSF test +# This tests the av1_tstosection5 bitstream filter that converts from +# MPEG-TS start code format to Section 5 (low overhead) format. +# The test generates an MPEG-TS file from IVF, then converts back using the BSF. +FATE_AV1_BSF-$(call ALLYES, IVF_DEMUXER MPEGTS_MUXER MPEGTS_DEMUXER AV1_PARSER AV1_TSTOSECTION5_BSF IVF_MUXER) += fate-av1-bsf-tstosection5 +fate-av1-bsf-tstosection5: SRC = $(TARGET_SAMPLES)/av1-test-vectors/av1-1-b8-05-mv.ivf +fate-av1-bsf-tstosection5: CMD = transcode ivf "$(SRC)" mpegts "-c:v copy" "-c:v copy -bsf:v av1_tstosection5 -f ivf" + +FATE_SAMPLES_FFMPEG += $(FATE_AV1_BSF-yes) + # H.264 read/write FATE_CBS_H264_CONFORMANCE_SAMPLES = \ diff --git a/tests/fate/mpegts.mak b/tests/fate/mpegts.mak index eaca8ec289..e768039aed 100644 --- a/tests/fate/mpegts.mak +++ b/tests/fate/mpegts.mak @@ -22,4 +22,19 @@ fate-mpegts-probe-pmt-merge: CMD = run $(PROBE_CODEC_NAME_COMMAND) -merge_pmt_ve FATE_SAMPLES_FFPROBE += $(FATE_MPEGTS_PROBE-yes) -fate-mpegts: $(FATE_MPEGTS_PROBE-yes) +# +# Test AV1 muxing/demuxing in MPEG-TS +# +# AV1 muxing: IVF -> MPEG-TS (Section 5 to start code format) +FATE_MPEGTS_AV1-$(call REMUX, MPEGTS, IVF_DEMUXER AV1_PARSER) += fate-mpegts-av1-mux +fate-mpegts-av1-mux: SRC = $(TARGET_SAMPLES)/av1-test-vectors/av1-1-b8-05-mv.ivf +fate-mpegts-av1-mux: CMD = framecrc -i "$(SRC)" -c:v copy -f mpegts + +# AV1 roundtrip: IVF -> MPEG-TS -> IVF (tests both muxing and demuxing) +FATE_MPEGTS_AV1-$(call REMUX, IVF MPEGTS, IVF_DEMUXER MPEGTS_DEMUXER AV1_PARSER AV1_TSTOSECTION5_BSF) += fate-mpegts-av1-roundtrip +fate-mpegts-av1-roundtrip: SRC = $(TARGET_SAMPLES)/av1-test-vectors/av1-1-b8-05-mv.ivf +fate-mpegts-av1-roundtrip: CMD = transcode ivf "$(SRC)" mpegts "-c:v copy" "-c:v copy -f ivf" + +FATE_SAMPLES_FFMPEG += $(FATE_MPEGTS_AV1-yes) + +fate-mpegts: $(FATE_MPEGTS_PROBE-yes) $(FATE_MPEGTS_AV1-yes) diff --git a/tests/ref/fate/av1-bsf-tstosection5 b/tests/ref/fate/av1-bsf-tstosection5 new file mode 100644 index 0000000000..06f51b0a64 --- /dev/null +++ b/tests/ref/fate/av1-bsf-tstosection5 @@ -0,0 +1,11 @@ +db041646a729b26618465bf2acb8b053 *tests/data/fate/av1-bsf-tstosection5.mpegts +57904 tests/data/fate/av1-bsf-tstosection5.mpegts +#tb 0: 1/90000 +#media_type 0: video +#codec_id 0: av1 +#dimensions 0: 352x288 +#sar 0: 1/1 +0, 0, 0, 3000, 30833, 0xfb97b07f, S=1, MPEGTS Stream ID, 1, 0x00bd00bd +0, 3000, 3000, 3000, 20779, 0xd71f36d5, F=0x0, S=1, MPEGTS Stream ID, 1, 0x00bd00bd +0, 6000, 6000, 3000, 3526, 0x643de3e4, F=0x0, S=1, MPEGTS Stream ID, 1, 0x00bd00bd +0, 9000, 9000, 3000, 5, 0x01920115, F=0x0, S=1, MPEGTS Stream ID, 1, 0x00bd00bd diff --git a/tests/ref/fate/mpegts-av1-mux b/tests/ref/fate/mpegts-av1-mux new file mode 100644 index 0000000000..2d39bcf9aa --- /dev/null +++ b/tests/ref/fate/mpegts-av1-mux @@ -0,0 +1,10 @@ +#extradata 0: 13, 0x0fc303f5 +#tb 0: 1/30 +#media_type 0: video +#codec_id 0: av1 +#dimensions 0: 352x288 +#sar 0: 1/1 +0, 0, 0, 1, 30833, 0xfb97b07f +0, 1, 1, 1, 20779, 0xd71f36d5, F=0x0 +0, 2, 2, 1, 3526, 0x643de3e4, F=0x0 +0, 3, 3, 1, 5, 0x01920115, F=0x0 diff --git a/tests/ref/fate/mpegts-av1-roundtrip b/tests/ref/fate/mpegts-av1-roundtrip new file mode 100644 index 0000000000..fbd8f76996 --- /dev/null +++ b/tests/ref/fate/mpegts-av1-roundtrip @@ -0,0 +1,11 @@ +db041646a729b26618465bf2acb8b053 *tests/data/fate/mpegts-av1-roundtrip.mpegts +57904 tests/data/fate/mpegts-av1-roundtrip.mpegts +#tb 0: 1/90000 +#media_type 0: video +#codec_id 0: av1 +#dimensions 0: 352x288 +#sar 0: 1/1 +0, 0, 0, 3000, 30842, 0x714eb082, S=1, MPEGTS Stream ID, 1, 0x00bd00bd +0, 3000, 3000, 3000, 20788, 0x180636d8, F=0x0, S=1, MPEGTS Stream ID, 1, 0x00bd00bd +0, 6000, 6000, 3000, 3532, 0x8002e3e6, F=0x0, S=1, MPEGTS Stream ID, 1, 0x00bd00bd +0, 9000, 9000, 3000, 11, 0x01d50117, F=0x0, S=1, MPEGTS Stream ID, 1, 0x00bd00bd -- 2.49.1
Hi, Le dim. 28 déc. 2025, 19:46, Jun Zhao via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch
This patch series V2 adds support for carrying AV1 video streams in MPEG-2 Transport Stream containers, following the AOM "Carriage of AV1 in MPEG-2 TS" specification.
Key Features
- MPEG-TS Muxer: Converts AV1 from Section 5 (low overhead) format to start code format, where each OBU is prefixed with a 0x000001 start code. Includes Registration Descriptor ('AV01') and AV1 Video Descriptor (0x80) in PMT. - MPEG-TS Demuxer: Identifies AV1 streams by stream_type 0x06 (private data) and Registration Descriptor. Outputs AV1 data in start code format. - av1_tstosection5 BSF: Converts AV1 from MPEG-TS start code format back to Section 5 format for compatibility with other containers and decoders. Automatically inserted when remuxing to MP4, MKV, FLV, IVF, and raw OBU files. - Decoder Support: Both libdav1d and libaom decoders can directly decode AV1 in start code format.
I haven't followed that AoM activity in a very long time, but from what I remember, there was a big reason why it remained a draft, as there was no consensus/decision possible for some of the features (maybe start code). I think the previously mentioned wisdom that ffmpeg shouldn't generate files that could end up not being spec-compliant, should be applied here as well. Ie maybe delay merging this until there is clear indication the implemented features are no longer subject to change. Note that for RTP, it was more about being lost in limbo, with little interop (ie between vendors) really possible, and with a global agreement on all classical points. I don't think what was merged in ffmpeg ended up non-spec compliant, but it could be argued it was out of luck. Mostly my 2 cents, Regards, Christophe
On 1/7/2026 10:25 AM, Christophe Gisquet via ffmpeg-devel wrote:
Hi,
Le dim. 28 déc. 2025, 19:46, Jun Zhao via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch
This patch series V2 adds support for carrying AV1 video streams in MPEG-2 Transport Stream containers, following the AOM "Carriage of AV1 in MPEG-2 TS" specification.
Key Features
- MPEG-TS Muxer: Converts AV1 from Section 5 (low overhead) format to start code format, where each OBU is prefixed with a 0x000001 start code. Includes Registration Descriptor ('AV01') and AV1 Video Descriptor (0x80) in PMT. - MPEG-TS Demuxer: Identifies AV1 streams by stream_type 0x06 (private data) and Registration Descriptor. Outputs AV1 data in start code format. - av1_tstosection5 BSF: Converts AV1 from MPEG-TS start code format back to Section 5 format for compatibility with other containers and decoders. Automatically inserted when remuxing to MP4, MKV, FLV, IVF, and raw OBU files. - Decoder Support: Both libdav1d and libaom decoders can directly decode AV1 in start code format.
I haven't followed that AoM activity in a very long time, but from what I remember, there was a big reason why it remained a draft, as there was no consensus/decision possible for some of the features (maybe start code).
I think the previously mentioned wisdom that ffmpeg shouldn't generate files that could end up not being spec-compliant, should be applied here as well. Ie maybe delay merging this until there is clear indication the implemented features are no longer subject to change.
Note that for RTP, it was more about being lost in limbo, with little interop (ie between vendors) really possible, and with a global agreement on all classical points. I don't think what was merged in ffmpeg ended up non-spec compliant, but it could be argued it was out of luck.
Mostly my 2 cents, Regards, Christophe
Muxing was removed from the PR, so that should not be an issue. Supposedly, the existing draft is being used by some content deliver services, but I'm not sure how widespread is support for it out there.
On Wed, Jan 7, 2026 at 9:26 PM Christophe Gisquet via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
Hi,
Le dim. 28 déc. 2025, 19:46, Jun Zhao via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch
This patch series V2 adds support for carrying AV1 video streams in MPEG-2 Transport Stream containers, following the AOM "Carriage of AV1 in MPEG-2 TS" specification.
Key Features
- MPEG-TS Muxer: Converts AV1 from Section 5 (low overhead) format to start code format, where each OBU is prefixed with a 0x000001 start code. Includes Registration Descriptor ('AV01') and AV1 Video Descriptor (0x80) in PMT. - MPEG-TS Demuxer: Identifies AV1 streams by stream_type 0x06 (private data) and Registration Descriptor. Outputs AV1 data in start code format. - av1_tstosection5 BSF: Converts AV1 from MPEG-TS start code format back to Section 5 format for compatibility with other containers and decoders. Automatically inserted when remuxing to MP4, MKV, FLV, IVF, and raw OBU files. - Decoder Support: Both libdav1d and libaom decoders can directly decode AV1 in start code format.
I haven't followed that AoM activity in a very long time, but from what I remember, there was a big reason why it remained a draft, as there was no consensus/decision possible for some of the features (maybe start code).
I think the previously mentioned wisdom that ffmpeg shouldn't generate files that could end up not being spec-compliant, should be applied here as well. Ie maybe delay merging this until there is clear indication the implemented features are no longer subject to change.
Note that for RTP, it was more about being lost in limbo, with little interop (ie between vendors) really possible, and with a global agreement on all classical points. I don't think what was merged in ffmpeg ended up non-spec compliant, but it could be argued it was out of luck.
Mostly my 2 cents, Regards, Christophe
My perspective is as follows: As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status. AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently.
Hi, Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
On Wed, Jan 7, 2026 at 9:26 PM
Mostly my 2 cents, Regards, Christophe
My perspective is as follows:
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
Ok. Sorry, I'm looking through a phone and just took the first hit for this topic. AV1 in TS is already being utilized in certain scenarios, and we also
need an implementation to validate this specification concurrently
Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point. My comment was not a NACK, but a call to caution and informed decision. If several people are taking that decision and judging the consequences can be handled, good. Regards
On Thu, Jan 8, 2026 at 3:32 PM Christophe Gisquet <christophe.gisquet@gmail.com> wrote:
Hi,
Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> a écrit :
On Wed, Jan 7, 2026 at 9:26 PM
Mostly my 2 cents, Regards, Christophe
My perspective is as follows:
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
Ok. Sorry, I'm looking through a phone and just took the first hit for this topic.
AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently
Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point.
My comment was not a NACK, but a call to caution and informed decision. If several people are taking that decision and judging the consequences can be handled, good.
As a non-native English speaker, my language may lack some politeness—please don't mind. This is not an argument. thx the comments.
Regards
-
On 1/8/26 08:32, Christophe Gisquet via ffmpeg-devel wrote:
Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point.
I also don't particularly like that argument, but what I do not understand is why we cannot get a public sample for sharing ? It seems to me this demuxer-case should require FF_COMPLIANCE_EXPERIMENTAL. Nicolas
On Fri, Jan 9, 2026 at 6:32 PM Nicolas Gaullier via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
On 1/8/26 08:32, Christophe Gisquet via ffmpeg-devel wrote:
Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point.
I also don't particularly like that argument, but what I do not understand is why we cannot get a public sample for sharing ?
It seems to me this demuxer-case should require FF_COMPLIANCE_EXPERIMENTAL.
I completely agree with this suggestion. In fact, I had thought about adding FF_COMPLIANCE_EXPERIMENTAL, but I unfortunately overlooked it during the many iterations.
On 1/9/2026 7:32 AM, Nicolas Gaullier via ffmpeg-devel wrote:
On 1/8/26 08:32, Christophe Gisquet via ffmpeg-devel wrote:
Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point.
I also don't particularly like that argument, but what I do not understand is why we cannot get a public sample for sharing ?
Kieran is trying to get one.
It seems to me this demuxer-case should require FF_COMPLIANCE_EXPERIMENTAL.
Why would a demuxer require the user to set that compliance value? Muxers i understand, but demuxers? Is there ever any reason you'd not want to have it output something it can handle?
On 1/9/26 16:42, James Almer via ffmpeg-devel wrote:
On 1/9/2026 7:32 AM, Nicolas Gaullier via ffmpeg-devel wrote:
On 1/8/26 08:32, Christophe Gisquet via ffmpeg-devel wrote:
Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point. I also don't particularly like that argument, but what I do not understand is why we cannot get a public sample for sharing ?
Kieran is trying to get one.
It seems to me this demuxer-case should require FF_COMPLIANCE_EXPERIMENTAL.
Why would a demuxer require the user to set that compliance value? Muxers i understand, but demuxers? Is there ever any reason you'd not want to have it output something it can handle?
I understand your point because we try to make demuxers robusts and tolerant to errors, compared to muxers which have to be very strictly compliants. The question is to know if you can be certain that the demuxer can actually handle it, with no pts issues/jerkiness or anything. I don't have information by myself, just read this thread and it is not reassuring, not comfortable I feel. For example, "there was no consensus/decision possible for some of the features (maybe start code)". Or "AV1 in TS is already being utilized in certain scenarios", but is it in some kind of "closed" environment or in the wild ? We already have scd and mmf demuxers using the compliance value for similar reasons, so I thought it could be appropriate for some months before the adoption get more widespread ? There is also another logic for rtpdec_vp9: a simple AV_LOG_WARNING. If something goes wrong, the user can realize he is doing something unusual and that may be the cause. Anyway, again, I have no information by myself, maybe I am wrong; my suggestion is to make the user aware of the status of the specs at this moment. Maybe at least rewording in the texi |"Carriage of AV1 in MPEG-2 TS <specification>" to "<working draft>" as recommended by the banner on aomediacodec.github.io ?| |Nicolas|
On Sat, Jan 10, 2026 at 1:20 AM Nicolas Gaullier via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
On 1/9/26 16:42, James Almer via ffmpeg-devel wrote:
On 1/9/2026 7:32 AM, Nicolas Gaullier via ffmpeg-devel wrote:
On 1/8/26 08:32, Christophe Gisquet via ffmpeg-devel wrote:
Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point. I also don't particularly like that argument, but what I do not understand is why we cannot get a public sample for sharing ?
Kieran is trying to get one.
It seems to me this demuxer-case should require FF_COMPLIANCE_EXPERIMENTAL.
Why would a demuxer require the user to set that compliance value? Muxers i understand, but demuxers? Is there ever any reason you'd not want to have it output something it can handle?
I understand your point because we try to make demuxers robusts and tolerant to errors, compared to muxers which have to be very strictly compliants.
The question is to know if you can be certain that the demuxer can actually handle it, with no pts issues/jerkiness or anything.
I don't have information by myself, just read this thread and it is not reassuring, not comfortable I feel.
For example, "there was no consensus/decision possible for some of the features (maybe start code)".
Or "AV1 in TS is already being utilized in certain scenarios", but is it in some kind of "closed" environment or in the wild ?
We already have scd and mmf demuxers using the compliance value for similar reasons, so I thought it could be appropriate for some months before the adoption get more widespread ?
There is also another logic for rtpdec_vp9: a simple AV_LOG_WARNING. If something goes wrong, the user can realize he is doing something unusual and that may be the cause.
Anyway, again, I have no information by myself, maybe I am wrong; my suggestion is to make the user aware of the status of the specs at this moment. Maybe at least rewording in the texi |"Carriage of AV1 in MPEG-2 TS <specification>" to "<working draft>" as recommended by the banner on aomediacodec.github.io ?|
|Nicolas|
Okay, adding a warning to the AV1 in TS demuxer and noting in the documentation that it's still in draft status is a good suggestion.
On Sat, Jan 10, 2026 at 1:20 AM Nicolas Gaullier via ffmpeg-devel <ffmpeg-devel@ffmpeg.org> wrote:
On 1/9/26 16:42, James Almer via ffmpeg-devel wrote:
On 1/9/2026 7:32 AM, Nicolas Gaullier via ffmpeg-devel wrote:
On 1/8/26 08:32, Christophe Gisquet via ffmpeg-devel wrote:
Le jeu. 8 janv. 2026, 02:24, mypopy--- via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> a écrit :
As James Almer pointed out in his response, the current merge request has already removed the muxer component, so there won't be any compatibility issues, even if the specification is still in draft status.
AV1 in TS is already being utilized in certain scenarios, and we also need an implementation to validate this specification concurrently Well, I don't particularly like that argument (flv extension again by some operator?), but this is beside the point. I also don't particularly like that argument, but what I do not understand is why we cannot get a public sample for sharing ?
Kieran is trying to get one.
It seems to me this demuxer-case should require FF_COMPLIANCE_EXPERIMENTAL.
Why would a demuxer require the user to set that compliance value? Muxers i understand, but demuxers? Is there ever any reason you'd not want to have it output something it can handle?
I understand your point because we try to make demuxers robusts and tolerant to errors, compared to muxers which have to be very strictly compliants.
The question is to know if you can be certain that the demuxer can actually handle it, with no pts issues/jerkiness or anything.
I don't have information by myself, just read this thread and it is not reassuring, not comfortable I feel.
For example, "there was no consensus/decision possible for some of the features (maybe start code)".
Or "AV1 in TS is already being utilized in certain scenarios", but is it in some kind of "closed" environment or in the wild ?
We already have scd and mmf demuxers using the compliance value for similar reasons, so I thought it could be appropriate for some months before the adoption get more widespread ?
There is also another logic for rtpdec_vp9: a simple AV_LOG_WARNING. If something goes wrong, the user can realize he is doing something unusual and that may be the cause.
Anyway, again, I have no information by myself, maybe I am wrong; my suggestion is to make the user aware of the status of the specs at this moment. Maybe at least rewording in the texi |"Carriage of AV1 in MPEG-2 TS <specification>" to "<working draft>" as recommended by the banner on aomediacodec.github.io ?|
The code has been updated according to this suggestion.
On 12/28/25 19:45, Jun Zhao via ffmpeg-devel wrote:
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch
Add functions to parse AV1 OBUs in MPEG-TS start code format (0x000001 prefix). This is needed for AV1 streams from MPEG-TS containers per AOM AV1 in MPEG-2 TS specification.
I have some questions around aligned PES and start codes; please correct me if I am wrong. Start codes: it seems to me current PR autodetects start codes, which means if a stream miss them, it would still be successfully decoded. This is possible because, additionally, AV1 requires aligned PES (which is unusual for h2645 typically). Being able to handle non-conformant files is nice until there are no downsides. The situation is maybe a little bit tricky here since this format is not widespread, you may want to be more carefull, check interoperability etc., and maybe you want to be warned that "this file may not be supported outside of ffmpeg" ? It seems there is no "issue" in the other way (a stream which is not pes-aligned would not be decodable since there is no "find_frame_end" in the parser to split the packets), but anyway maybe it could be an interesting opportunity to introduce pes alignment support in mpegts ? If I understand correctly, it would allow disabling packet repacking in some use cases (j2k for example: I just had a quick look today since mpegts mandates pes alignment for it too). Nicolas
On 1/12/2026 1:31 PM, Nicolas Gaullier via ffmpeg-devel wrote:
On 12/28/25 19:45, Jun Zhao via ffmpeg-devel wrote:
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch
Add functions to parse AV1 OBUs in MPEG-TS start code format (0x000001 prefix). This is needed for AV1 streams from MPEG-TS containers per AOM AV1 in MPEG-2 TS specification.
I have some questions around aligned PES and start codes; please correct me if I am wrong.
Start codes: it seems to me current PR autodetects start codes, which means if a stream miss them, it would still be successfully decoded. This is possible because, additionally, AV1 requires aligned PES (which
Where does the spec say that?
is unusual for h2645 typically). Being able to handle non-conformant files is nice until there are no downsides. The situation is maybe a little bit tricky here since this format is not widespread, you may want to be more carefull, check interoperability etc., and maybe you want to be warned that "this file may not be supported outside of ffmpeg" ? It
Why would we warn that? Demuxing has no need for any kind of warning like that. We're reading a file and trying to make sense of its contents. What other readers can or can't do is irrelevant.
seems there is no "issue" in the other way (a stream which is not pes- aligned would not be decodable since there is no "find_frame_end" in the parser to split the packets), but anyway maybe it could be
I already wrote an implementation for Temporal Unit assembly in av1_parser, but I'm waiting until all is said and done, and probably an actual sample is provided.
an interesting opportunity to introduce pes alignment support in mpegts ? If I understand correctly, it would allow disabling packet repacking in some use cases (j2k for example: I just had a quick look today since mpegts mandates pes alignment for it too).
Nicolas
_______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
On 1/12/26 22:05, James Almer via ffmpeg-devel wrote:
On 1/12/2026 1:31 PM, Nicolas Gaullier via ffmpeg-devel wrote:
On 12/28/25 19:45, Jun Zhao via ffmpeg-devel wrote:
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch
Add functions to parse AV1 OBUs in MPEG-TS start code format (0x000001 prefix). This is needed for AV1 streams from MPEG-TS containers per AOM AV1 in MPEG-2 TS specification.
I have some questions around aligned PES and start codes; please correct me if I am wrong.
Start codes: it seems to me current PR autodetects start codes, which means if a stream miss them, it would still be successfully decoded. This is possible because, additionally, AV1 requires aligned PES (which
Where does the spec say that?
3.4 : "All the PES shall have data_alignment_indicator set to 1. Usage of *data_stream_alignment_descriptor* is not specified and the only allowed *alignment_type* is 1 (Access unit level)." But anyway, at the end, it does not matter if packet repacking is to be implemented just afterwards as you say (if I catch it correctly, it will be required for muxing).
like that. We're reading a file and trying to make sense of its contents. What other readers can or can't do is irrelevant.
I understand your point, that makes it clear and it is ok if everybody is comfortable with that.
I already wrote an implementation for Temporal Unit assembly in av1_parser, but I'm waiting until all is said and done, and probably an actual sample is provided.
This makes me think of a latest very specific question but again, I could be mistaken - correct me if am wrong - after this PR 21307, an out-of-spec stream with missing start codes will be playable (frames split at PES boundaries), but later on, once packet repacking implemented, it wont be (frames split by looking for start codes) ? In case I get this right, maybe the future merge of PR 21307 would have to be delayed until your complementary work is also ready to be merged at the same time ? Nicolas
On 1/13/2026 6:50 AM, Nicolas Gaullier via ffmpeg-devel wrote:
On 1/12/26 22:05, James Almer via ffmpeg-devel wrote:
On 1/12/2026 1:31 PM, Nicolas Gaullier via ffmpeg-devel wrote:
On 12/28/25 19:45, Jun Zhao via ffmpeg-devel wrote:
PR #21307 opened by Jun Zhao (mypopydev) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21307.patch
Add functions to parse AV1 OBUs in MPEG-TS start code format (0x000001 prefix). This is needed for AV1 streams from MPEG-TS containers per AOM AV1 in MPEG-2 TS specification.
I have some questions around aligned PES and start codes; please correct me if I am wrong.
Start codes: it seems to me current PR autodetects start codes, which means if a stream miss them, it would still be successfully decoded. This is possible because, additionally, AV1 requires aligned PES (which
Where does the spec say that?
3.4 : "All the PES shall have data_alignment_indicator set to 1. Usage of *data_stream_alignment_descriptor* is not specified and the only allowed *alignment_type* is 1 (Access unit level)."
But anyway, at the end, it does not matter if packet repacking is to be implemented just afterwards as you say (if I catch it correctly, it will be required for muxing).
like that. We're reading a file and trying to make sense of its contents. What other readers can or can't do is irrelevant.
I understand your point, that makes it clear and it is ok if everybody is comfortable with that.
I already wrote an implementation for Temporal Unit assembly in av1_parser, but I'm waiting until all is said and done, and probably an actual sample is provided.
This makes me think of a latest very specific question but again, I could be mistaken - correct me if am wrong - after this PR 21307, an out-of-spec stream with missing start codes will be playable (frames split at PES boundaries), but later on, once packet repacking implemented, it wont be (frames split by looking for start codes) ? In case I get this right, maybe the future merge of PR 21307 would have to be delayed until your complementary work is also ready to be merged at the same time ?
Yes, it will be part of the PR. The current code in the PR exports each frame separately, exactly as it was muxed, when an entire TU should be exported instead. That's what the parser will handle.
Nicolas
_______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
participants (5)
-
Christophe Gisquet -
James Almer -
Jun Zhao -
mypopy＠gmail.com -
Nicolas Gaullier