[FFmpeg-devel] [PATCH 1/8] avutil: introduce an Immersive Audio Model and Formats API

[Anton Khirnov]

Quoting James Almer (2023-12-05 23:43:55)
> diff --git a/libavutil/iamf.h b/libavutil/iamf.h
> new file mode 100644
> index 0000000000..bc0363153d
> --- /dev/null
> +++ b/libavutil/iamf.h
> +/**
> + * @file
> + * Immersive Audio Model and Formats API header
> + * @see <a href="https://aomediacodec.github.io/iamf/">Immersive Audio Model and Formats</a>
> + */
> +
> +#include <stdint.h>
> +#include <stddef.h>
> +
> +#include "attributes.h"
> +#include "avassert.h"
> +#include "channel_layout.h"
> +#include "dict.h"
> +#include "rational.h"
> +
> +/**
> + * @defgroup lavf_iamf_params Parameter Definition
> + * @{
> + * Parameters as defined in section 3.6.1 and 3.8 of IAMF.
> + * @}
> + * @defgroup lavf_iamf_audio Audio Element
> + * @{
> + * Audio Elements as defined in section 3.6 of IAMF.
> + * @}
> + * @defgroup lavf_iamf_mix Mix Presentation
> + * @{
> + * Mix Presentations as defined in section 3.7 of IAMF.
> + * @}
> + *
> + * @}
> + * @addtogroup lavf_iamf_params
> + * @{
> + */
> +enum AVIAMFAnimationType {
> +    AV_IAMF_ANIMATION_TYPE_STEP,
> +    AV_IAMF_ANIMATION_TYPE_LINEAR,
> +    AV_IAMF_ANIMATION_TYPE_BEZIER,
> +};
> +
> +/**
> + * Mix Gain Parameter Data as defined in section 3.8.1 of IAMF.
> + *
> + * Subblocks in AVIAMFParamDefinition use this struct when the value or
> + * @ref AVIAMFParamDefinition.param_definition_type param_definition_type is
> + * AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN.

This is the wrong place for this second paragraph. IMO it should go into
doxy for either param_definition_type, or
AV_IAMF_PARAMETER_DEFINITION_MIX_GAIN.

> +/**
> + * Parameters as defined in section 3.6.1 of IAMF.

It's not immediately obvious how the contents are structured, so I'd
extend it with something like:

The struct is allocated along with an array of subblocks, which are
either AVIAMFMixGain, AVIAMFDemixingInfo, or AVIAMFReconGain depending
on the value of param_definition_type. This array is placed
subblocks_offset bytes after the start of this struct.

> + */
> +typedef struct AVIAMFParamDefinition {
> +    const AVClass *av_class;
> +
> +    size_t subblocks_offset;

Offset in bytes from the start of this struct, at which the subblocks
array is located.

> +    size_t subblock_size;

Size in bytes of each element in the subblocks array.

> +
> +    enum AVIAMFParamDefinitionType param_definition_type;

This struct is already called ParamDefinition, so this could be just
type. Also:

Parameters type. Determines the type of the subblock elements.

> +    unsigned int nb_subblocks;

Number of subblocks in the array.

> +
> +    unsigned int parameter_id;
> +    unsigned int parameter_rate;
> +    unsigned int param_definition_mode;
> +    unsigned int duration;
> +    unsigned int constant_subblock_duration;

These should be documented.

> +typedef struct AVIAMFLayer {
> +    const AVClass *av_class;
> +
> +    AVChannelLayout ch_layout;
> +
> +    /**
> +     * A bitmask which may contain a combination of AV_IAMF_LAYER_FLAG_* flags.
> +     */
> +    unsigned int flags;
> +    /**
> +     * Output gain channel flags as defined in section 3.6.2 of IAMF.
> +     *
> +     * This field is defined only if @ref AVIAMFAudioElement.audio_element_type
> +     * "the parent's Audio Element type" is AV_IAMF_AUDIO_ELEMENT_TYPE_CHANNEL,
> +     * must be 0 otherwise.
> +     */
> +    unsigned int output_gain_flags;
> +    /**
> +     * Output gain as defined in section 3.6.2 of IAMF.
> +     *
> +     * Must be 0 if @ref output_gain_flags is 0.
> +     */
> +    AVRational output_gain;
> +    /**
> +     * Ambisonics mode as defined in section 3.6.3 of IAMF.
> +     *
> +     * This field is defined only if @ref AVIAMFAudioElement.audio_element_type
> +     * "the parent's Audio Element type" is AV_IAMF_AUDIO_ELEMENT_TYPE_SCENE.
> +     *
> +     * If AV_IAMF_AMBISONICS_MODE_MONO, channel_mapping is defined implicitly
> +     * (Ambisonic Order) or explicitly (Custom Order with ambi channels) in
> +     * @ref ch_layout.
> +     * If AV_IAMF_AMBISONICS_MODE_PROJECTION, @ref demixing_matrix must be set.
> +     */
> +    enum AVIAMFAmbisonicsMode ambisonics_mode;
> +
> +    /**
> +     * Demixing matrix as defined in section 3.6.3 of IAMF.
> +     *
> +     * May be set only if @ref ambisonics_mode == AV_IAMF_AMBISONICS_MODE_PROJECTION,
> +     * must be NULL otherwise.
> +     */
> +    AVRational *demixing_matrix;

How many elements are in the array?

-- 
Anton Khirnov