[FFmpeg-devel] [PATCH] avcodec/vp9block: fix runtime error: signed integer overflow: 196675 * 20670 cannot be represented in type 'int'

Michael Niedermayer michael at niedermayer.cc
Sun May 21 13:51:56 EEST 2017


On Sat, May 20, 2017 at 10:35:34PM -0400, Ronald S. Bultje wrote:
> Hi,
> 
> On Sat, May 20, 2017 at 8:12 PM, Michael Niedermayer <michael at niedermayer.cc
> > wrote:
> 
> > Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624
> >
> > Found-by: continuous fuzzing process https://github.com/google/oss-
> > fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > ---
> >  libavcodec/vp9block.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/libavcodec/vp9block.c b/libavcodec/vp9block.c
> > index ae2f0e4c6f..a16ccdccdb 100644
> > --- a/libavcodec/vp9block.c
> > +++ b/libavcodec/vp9block.c
> > @@ -915,9 +915,9 @@ skip_eob:
> >          if (!--band_left)
> >              band_left = band_counts[++band];
> >          if (is_tx32x32)
> > -            STORE_COEF(coef, rc, ((vp8_rac_get(c) ? -val : val) *
> > qmul[!!i]) / 2);
> > +            STORE_COEF(coef, rc, (int)((vp8_rac_get(c) ? -val : val) *
> > (unsigned)qmul[!!i]) / 2);
> >          else
> > -            STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) *
> > qmul[!!i]);
> > +            STORE_COEF(coef, rc, (vp8_rac_get(c) ? -val : val) *
> > (unsigned)qmul[!!i]);
> >          nnz = (1 + cache[nb[i][0]] + cache[nb[i][1]]) >> 1;
> >          tp = p[band][nnz];
> >      } while (++i < n_coeffs);
> > --
> > 2.13.0
> 
> 
> Since this is the only use of qmul[], why don't you make the array unsigned
> instead? That saves a cast.

will change and apply

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Dictatorship naturally arises out of democracy, and the most aggravated
form of tyranny and slavery out of the most extreme liberty. -- Plato
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170521/cd180c44/attachment.sig>


More information about the ffmpeg-devel mailing list