[FFmpeg-devel] [PATCH] avcodec/cbs_h2645: Avoid function pointer casts, fix UB

[Mark Thompson]

On 25/02/2024 01:55, Andreas Rheinhardt wrote:
> The SEI message read/write functions are called
> via function pointers where the SEI message-specific
> context is passed as void*. But the actual function
> definitions use a pointer to their proper context
> in place of void*, making the calls undefined behaviour.
> Clang UBSan 17 warns about this.
> 
> This commit fixes this by adding wrapper functions
> (created via macros) that have the right type that
> call the actual functions. This reduced the number of failing
> FATE tests with UBSan from 164 to 85 here.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
> ---
>   libavcodec/cbs_h2645.c                | 15 +++++++
>   libavcodec/cbs_h264_syntax_template.c | 35 ++++++++--------
>   libavcodec/cbs_h265_syntax_template.c | 58 +++++++++++++--------------
>   libavcodec/cbs_h266_syntax_template.c |  8 ++--
>   libavcodec/cbs_sei.h                  |  7 ----
>   libavcodec/cbs_sei_syntax_template.c  | 47 +++++++++++-----------
>   6 files changed, 88 insertions(+), 82 deletions(-)
> 
> diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c
> index 2fb249bcd3..8e4af7b2cc 100644
> --- a/libavcodec/cbs_h2645.c
> +++ b/libavcodec/cbs_h2645.c
> @@ -235,6 +235,16 @@ static int cbs_h265_payload_extension_present(GetBitContext *gbc, uint32_t paylo
>   #define FUNC_H266(name) FUNC_NAME1(READWRITE, h266, name)
>   #define FUNC_SEI(name)  FUNC_NAME1(READWRITE, sei,  name)
>   
> +#define SEI_FUNC(name, args) \
> +static int FUNC(name) args;  \
> +static int FUNC(name ## _internal)(CodedBitstreamContext *ctx, \
> +                                   RWContext *rw, void *cur,   \
> +                                   SEIMessageState *state)     \
> +{ \
> +    return FUNC(name)(ctx, rw, cur, state); \
> +} \
> +static int FUNC(name) args
> +
>   #define SUBSCRIPTS(subs, ...) (subs > 0 ? ((int[subs + 1]){ subs, __VA_ARGS__ }) : NULL)
>   
>   #define u(width, name, range_min, range_max) \
> @@ -2070,6 +2080,11 @@ const CodedBitstreamType ff_cbs_type_h266 = {
>       .close             = &cbs_h266_close,
>   };
>   
> +// Macro for the read/write pair.
> +#define SEI_MESSAGE_RW(codec, name) \
> +    .read  = cbs_ ## codec ## _read_  ## name ## _internal, \
> +    .write = cbs_ ## codec ## _write_ ## name ## _internal
> +
>   static const SEIMessageTypeDescriptor cbs_sei_common_types[] = {
>       {
>           SEI_TYPE_FILLER_PAYLOAD,

LGTM, thank you.

- Mark