[FFmpeg-devel] [PATCH v2 00/18] Add timed metadata tracks, track references, channel layout

[Erkki Seppälä]

From: Erkki Seppälä <erkki.seppala at vincit.fi>

Hello, developers!

Here is a patch set version 2 with the goal of adding the following
features to the ISO base media file format support of FFmpeg (also
available at
https://github.com/nokiatech/FFmpeg/tree/timed-metadata-v2):

The branch is still based on the old master repository version to make
it easier to compare. It seems to rebase on top of current master
without issues, though. Except for the changelog that is..

The differences to the version 1 of the patch set are broadly (most
based of review feedback, thanks for that!):
- Ensure that fate passes (sorry about that..)
- Dropped the integrated patch "libavformat/utils: added ability to
  probe AVMEDIA_TYPE_DATA format"
- Dropped the patch to add av_arraydup
- Dropped the patch to add AVData in favor of using AVFrame
- Dropped the 'lib' prefixes from patch descriptions
- Better comments
- Use trailing comma in enumerations
- Some of the structs required for side data had their variable
  length data replaced with fixed length representations.
- Some of the structs required for side data had the implicit
  0-element array after the struct replaced with a builtin struct
  of an 1-element array (that is to be overflown)
- Side data is accessed more carefully in presence of
  misaligned data. Generated site data is well-aligned.
- Writing the proper channel number count to the ftyp box is now
  guarded by the adding of the chnl box. The chnl box is difficult to
  parse without to parse without the correct number here and is
  explicitly required by the standard in this case.
- Squashed the patch adding codec id for AV_CODEC_ID_META, the metacodec
  and the patch for adding the metacodec to the build (and codec list)
- Metacodec encoder/decoder now uses reference counted buffers
- Added support for avcodec_{send,receive}_{frame,packet} as the
  proper public API to use the metacodec. (Though the metacodec itself
  still provides only the old API which the new one then uses.)
- Fixed some cases of handling AVMEDIA_TYPE_DATA too early in the
  if-chains that broke down handling of tmcd and rtp in movenc.c
- After consulting the standard with an expert, it was again noticed
  that there can indeed be multiple kinds of trefs inside the one tref
  box of isom. The patches were reworked though to simplify the
  changes needed to introduce the multiple tref types and multiple
  track references.
- Also the reading of track references support multiple track
  references and multiple kinds of track references
- Introduced enumerations for ISO/IEC 23001-8 for channel layouts
- Reworked the channel layouts to use only one side data with
  a more complex struct

Erkki Seppälä (18):
  avformat/movenc: added ability to use original stream ids as track ids
    instead of regenerating them
  avformat/movenc: mov_write_ftyp_tag: write the major brand as a
    compatible brand
  avcodec/metaenc: added an encoder/decoder for timed metadata
  avformat/movenc: deal with AVMEDIA_TYPE_DATA by using AV_CODEC_ID_META
  avformat/movenc: support for multiple and client-provided track
    references
  avformat/movenc: add the urim sample descriptor
  avformat/movenc, isom: support metadata in mp4 files
  avcodec/utils: do_decode now supports AVMEDIA_TYPE_DATA
  avcodec/utils: do_encode now supports AVMEDIA_TYPE_DATA
  avformat/mov, isom: read (multiple) track references (tag and multiple
    ids)
  avformat/mov: basic support for identifying (and reading) timed
    metadata
  avformat/mov: ff_mov_read_stsd_entries now deals with
    AVMEDIA_TYPE_DATA
  avformat/mov: read urim metadata from meta
  avcodec/avcodec, avformat/movenc: support embedding channel layout to
    stream side data
  avformat/movenc: mov_write_audio_tag can write the proper number of
    channels, not the hardcoded 2
  avcodec/avcodec, avformat/movenc: introduced
    AV_PKT_DATA_TRACK_ALTERNATE_GROUP side data for expressing alternate
    groups
  doc/examples/extract_timed_metadata: added a bare-bones metadata
    extractor; find only the frames
  doc/examples/muxing_with_metadata: example for dealing with timed meta
    data

 configure                                |   4 +-
 doc/examples/Makefile                    |   3 +
 doc/examples/extract_timed_metadata.c    | 233 ++++++++
 doc/examples/muxing_with_metadata.c      | 885 +++++++++++++++++++++++++++++++
 libavcodec/Makefile                      |   2 +-
 libavcodec/allcodecs.c                   |   3 +
 libavcodec/avcodec.h                     | 119 ++++-
 libavcodec/codec_desc.c                  |   8 +
 libavcodec/metacodec.c                   |  95 ++++
 libavcodec/utils.c                       |   6 +
 libavformat/Makefile                     |   4 +-
 libavformat/isom.c                       |   7 +
 libavformat/isom.h                       |   4 +
 libavformat/mov.c                        | 288 +++++++++-
 libavformat/movenc.c                     | 378 +++++++++++--
 libavformat/movenc.h                     |   8 +-
 libavformat/movmeta.h                    |  46 ++
 libavformat/movtref.c                    | 115 ++++
 libavformat/movtref.h                    |  75 +++
 libavutil/channel_layout_isoiec23001_8.h |  97 ++++
 20 files changed, 2328 insertions(+), 52 deletions(-)
 create mode 100644 doc/examples/extract_timed_metadata.c
 create mode 100644 doc/examples/muxing_with_metadata.c
 create mode 100644 libavcodec/metacodec.c
 create mode 100644 libavformat/movmeta.h
 create mode 100644 libavformat/movtref.c
 create mode 100644 libavformat/movtref.h
 create mode 100644 libavutil/channel_layout_isoiec23001_8.h

-- 
2.7.4