[FFmpeg-devel] [PATCH 05/10] diractab: expose the maximum quantization index as a macro

[Rostislav Pehlivanov]

On 24 June 2016 at 02:36, Michael Niedermayer <michael at niedermayer.cc>
wrote:

> On Thu, Jun 23, 2016 at 06:06:59PM +0100, Rostislav Pehlivanov wrote:
> > Prevents having to have random magic values in the decoder and a
> > separate macro in the encoder.
> >
> > Signed-off-by: Rostislav Pehlivanov <rpehlivanov at obe.tv>
> > ---
> >  libavcodec/diracdec.c | 8 ++++----
> >  libavcodec/diractab.h | 2 ++
> >  libavcodec/vc2enc.c   | 9 +++------
> >  3 files changed, 9 insertions(+), 10 deletions(-)
> >
> > diff --git a/libavcodec/diracdec.c b/libavcodec/diracdec.c
> > index c8ab2df..48ba194 100644
> > --- a/libavcodec/diracdec.c
> > +++ b/libavcodec/diracdec.c
> > @@ -486,7 +486,7 @@ static inline void codeblock(DiracContext *s,
> SubBand *b,
> >          b->quant = quant;
> >      }
> >
> > -    if (b->quant > 115) {
> > +    if (b->quant > DIRAC_MAX_QUANT_INDEX) {
> >          av_log(s->avctx, AV_LOG_ERROR, "Unsupported quant %d\n",
> b->quant);
> >          b->quant = 0;
> >          return;
> > @@ -676,12 +676,12 @@ static void decode_subband(DiracContext *s,
> GetBitContext *gb, int quant,
> >      uint8_t *buf2 = b2 ? b2->ibuf + top * b2->stride: NULL;
> >      int x, y;
> >
> > -    if (quant > 115) {
> > +    if (quant > DIRAC_MAX_QUANT_INDEX) {
> >          av_log(s->avctx, AV_LOG_ERROR, "Unsupported quant %d\n", quant);
> >          return;
> >      }
>
> > -    qfactor = ff_dirac_qscale_tab[quant & 0x7f];
> > -    qoffset = ff_dirac_qoffset_intra_tab[quant & 0x7f] + 2;
> > +    qfactor = ff_dirac_qscale_tab[quant & DIRAC_MAX_QUANT_INDEX];
> > +    qoffset = ff_dirac_qoffset_intra_tab[quant & DIRAC_MAX_QUANT_INDEX]
> + 2;
>
> if iam not missing anything then
> DIRAC_MAX_QUANT_INDEX / FF_ARRAY_ELEMS(ff_dirac_qscale_tab) is
> 116
> quant & 116 looks unintended
>
>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> I know you won't believe me, but the highest form of Human Excellence is
> to question oneself and others. -- Socrates
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>
That AND was originally protection for invalid quantization indices causing
an out of bounds access.
Forgot to remove them, changed locally.