[FFmpeg-devel] [PATCH v7 1/8] Add generic metadata injection using AV_PKT_DATA_METADATA_UPDATE

[Romain Beauxis]

Le lun. 10 mars 2025 à 02:13, Andreas Rheinhardt
<andreas.rheinhardt at outlook.com> a écrit :
>
> Romain Beauxis:
> > libavcodec/decode.c: intercept `AV_PKT_DATA_METADATA_UPDATE` packet
> > extra data, attach them to the next decoded frame.
> >
> > ---
> >  libavcodec/decode.c | 20 ++++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> >
> > diff --git a/libavcodec/decode.c b/libavcodec/decode.c
> > index cac7e620d2..96e2f0ce95 100644
> > --- a/libavcodec/decode.c
> > +++ b/libavcodec/decode.c
> > @@ -97,6 +97,8 @@ typedef struct DecodeContext {
> >      int lcevc_frame;
> >      int width;
> >      int height;
> > +
> > +    AVDictionary *pending_metadata;
> >  } DecodeContext;
> >
> >  static DecodeContext *decode_ctx(AVCodecInternal *avci)
> > @@ -729,6 +731,8 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacke
> >  {
> >      AVCodecInternal *avci = avctx->internal;
> >      DecodeContext     *dc = decode_ctx(avci);
> > +    const uint8_t *side_metadata;
> > +    size_t size;
> >      int ret;
> >
> >      if (!avcodec_is_open(avctx) || !av_codec_is_decoder(avctx->codec))
> > @@ -746,6 +750,14 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacke
> >          ret = av_packet_ref(avci->buffer_pkt, avpkt);
> >          if (ret < 0)
> >              return ret;
> > +
> > +        side_metadata = av_packet_get_side_data(avpkt, AV_PKT_DATA_METADATA_UPDATE, &size);
> > +        if (side_metadata) {
> > +            av_dict_free(&dc->pending_metadata);
> > +            ret = av_packet_unpack_dictionary(side_metadata, size, &dc->pending_metadata);
> > +            if (ret < 0)
> > +                return ret;
> > +        }
> >      } else
> >          dc->draining_started = 1;
> >
> > @@ -815,6 +827,7 @@ fail:
> >  int ff_decode_receive_frame(AVCodecContext *avctx, AVFrame *frame)
> >  {
> >      AVCodecInternal *avci = avctx->internal;
> > +    DecodeContext     *dc = decode_ctx(avci);
> >      int ret;
> >
> >      if (!avcodec_is_open(avctx) || !av_codec_is_decoder(avctx->codec))
> > @@ -887,6 +900,12 @@ int ff_decode_receive_frame(AVCodecContext *avctx, AVFrame *frame)
> >          }
> >      }
> >  #endif
> > +
> > +    if (dc->pending_metadata) {
> > +        av_dict_copy(&frame->metadata, dc->pending_metadata, AV_DICT_APPEND);
> > +        av_dict_free(&dc->pending_metadata);
> > +    }
> > +
> >      return 0;
> >  fail:
> >      av_frame_unref(frame);
> > @@ -2314,4 +2333,5 @@ void ff_decode_internal_uninit(AVCodecContext *avctx)
> >      DecodeContext *dc = decode_ctx(avci);
> >
> >      av_refstruct_unref(&dc->lcevc);
> > +    av_dict_free(&dc->pending_metadata);
> >  }
>
> Why is this not in ff_decode_frame_props_from_pkt() (in
> add_metadata_from_side_data())?

At the moment, metadata updates arrive attached to ogg header packets.
See this test output:

Stream ID: 0, packet PTS: 704, packet DTS: 704, metadata:
encoder=Lavc61.19.100 libvorbis:title=First Stream
Stream ID: 0, frame PTS: 704, metadata:
Stream ID: 0, packet PTS: 0, packet DTS: 0, metadata:
encoder=Lavc61.19.100 libvorbis:title=First Stream
Stream ID: 0, packet PTS: 0, packet DTS: 0, metadata:
encoder=Lavc61.19.100 libvorbis:title=Second Stream
Stream ID: 0, packet PTS: 0, packet DTS: 0, metadata:
encoder=Lavc61.19.100 libvorbis:title=Second Stream
Stream ID: 0, packet PTS: 0, packet DTS: 0, metadata:
encoder=Lavc61.19.100 libvorbis:title=Second Stream
Stream ID: 0, frame PTS: 0, metadata: encoder=Lavc61.19.100
libvorbis:title=Second Stream

(Note that this is the current behavior)

These packets do not provide an audio frame immediately so the
metadata update has to be delayed to the next decoded frame.

It seems to me that the semantics of these functions is to get data
from a submitted packet and attach it to its corresponding frame.

In this case this would not work.

In future work, we want to remove the ogg header packet from the
demuxer output and attach the metadata update directly to the first
packet sent for decoding.

At that point, the functionality could be moved to the
add_metadata_from_side_data function.

Does that make sense?

-- Romain