[FFmpeg-devel] [PATCH] avcodec/vp9block: fix runtime error: signed integer overflow: 196675 * 20670 cannot be represented in type 'int'

Ronald S. Bultje rsbultje at gmail.com
Sun May 21 14:25:10 EEST 2017


Hi,

On Sun, May 21, 2017 at 6:57 AM, Michael Niedermayer <michael at niedermayer.cc
> wrote:

> On Sun, May 21, 2017 at 12:51:56PM +0200, Michael Niedermayer wrote:
> > On Sat, May 20, 2017 at 10:35:34PM -0400, Ronald S. Bultje wrote:
> > > Hi,
> > >
> > > On Sat, May 20, 2017 at 8:12 PM, Michael Niedermayer
> <michael at niedermayer.cc
> > > > wrote:
> > >
> > > > Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624
> > > >
> > > > Found-by: continuous fuzzing process https://github.com/google/oss-
> > > > fuzz/tree/master/projects/ffmpeg
> > > > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > > > ---
> > > >  libavcodec/vp9block.c | 4 ++--
> > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c
> > > > index ae2f0e4c6f..a16ccdccdb 100644
> > > > --- a/libavcodec/vp9block.c
> > > > +++ b/libavcodec/vp9block.c
> > > > @@ -915,9 +915,9 @@ skip_eob:
> > > >          if (!--band_left)
> > > >              band_left = band_counts[++band];
> > > >          if (is_tx32x32)
> > > > -            STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) *
> > > > qmul[!!i]) / 2);
> > > > +            STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val :
> val) *
> > > > (unsigned)qmul[!!i]) / 2);
> > > >          else
> > > > -            STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) *
> > > > qmul[!!i]);
> > > > +            STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) *
> > > > (unsigned)qmul[!!i]);
> > > >          nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1;
> > > >          tp = p[band][nnz];
> > > >      } while (++i < n_coeffs);
> > > > --
> > > > 2.13.0
> > >
> > >
> > > Since this is the only use of qmul[], why don't you make the array
> unsigned
> > > instead? That saves a cast.
> >
> > will change and apply
>
> actually, no i cant do that
> qmul is int16_t *, uint16_t * will be "promoted" to signed int
>
> do you see another way ?


Hm, crap, you're right. OK, existing patch is fine. (Changing type to
unsigned is also fine, but I probably slightly prefer the cast so the
in-memory representation is smaller.)

Ronald


More information about the ffmpeg-devel mailing list