[FFmpeg-devel] [PATCH 1/4] avcodec/vvc/vvc_ps: Check before access

[Andreas Rheinhardt]

Andreas Rheinhardt:
> max_bin_idx can be at most LMCS_MAX_BIN_SIZE - 1 here,
> so pivot[LCMS_MAX_BIN_SIZE + 1] may be accessed,
> but pivot has only LCMS_MAX_BIN_SIZE + 1 elements
> (unless the values of pivot were so that it is always
> assured that pivot[LCMS_MAX_BIN_SIZE] is always < sample
> (which it is iff it is always < 2^bit_depth - 1)).
> So reorder the checks.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
> ---
> I don't know whether this can be triggered at all.
> 
>  libavcodec/vvc/vvc_ps.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/vvc/vvc_ps.c b/libavcodec/vvc/vvc_ps.c
> index 53a86321d6..376027ed81 100644
> --- a/libavcodec/vvc/vvc_ps.c
> +++ b/libavcodec/vvc/vvc_ps.c
> @@ -652,7 +652,7 @@ static int lmcs_derive_lut(VVCLMCS *lmcs, const H266RawAPS *rlmcs, const H266Raw
>      i = lmcs->min_bin_idx;
>      for (uint16_t sample = 0; sample < max; sample++) {
>          uint16_t inv_sample;
> -        while (sample >= lmcs->pivot[i + 1] && i <= lmcs->max_bin_idx)
> +        while (i <= lmcs->max_bin_idx && sample >= lmcs->pivot[i + 1])
>              i++;
>  
>          inv_sample = lmcs_derive_lut_sample(sample, input_pivot, lmcs->pivot,

Will apply this patchset tomorrow unless there are objections.

- Andreas