[FFmpeg-devel] [PATCH] avutil/video_enc_param: fix warning

[Nicolas George]

lance.lmwang at gmail.com (12020-05-21):
> From: Limin Wang <lance.lmwang at gmail.com>
> 
> warning: comparison is always false due to limited range of data type [-Wtype-limits]

> Also nb_blocks is unsigned int, so nb_blocks * sizeof(AVVideoBlockParams) may overflow,
> so force to size_t

No it may not, the test just before prevents it.

> 
> Signed-off-by: Limin Wang <lance.lmwang at gmail.com>
> ---
>  libavutil/video_enc_params.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c
> index c46c0f1..4a4c85f 100644
> --- a/libavutil/video_enc_params.c
> +++ b/libavutil/video_enc_params.c
> @@ -33,8 +33,8 @@ AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type,
>      size_t size;
>  
>      size = sizeof(*par);

> -    if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) ||
> +    if (nb_blocks > UINT_MAX / sizeof(AVVideoBlockParams) ||

These tests are not equivalent.

> -        nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size)
> +        (size_t)nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size)

The cast is unnecessary due to C's promotion rules.

>          return NULL;
>      size += sizeof(AVVideoBlockParams) * nb_blocks;
>  

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200521/54726724/attachment.sig>