[FFmpeg-devel] [PATCH] mjpegdec: consider chroma subsampling in size check

[Michael Niedermayer]

On Fri, Dec 04, 2015 at 03:14:21PM +0100, Andreas Cadhalpun wrote:
> On 03.12.2015 15:48, Michael Niedermayer wrote:
> > On Wed, Dec 02, 2015 at 10:00:13PM +0100, Andreas Cadhalpun wrote:
> >> @@ -1293,14 +1296,16 @@ static int mjpeg_decode_scan(MJpegDecodeContext *s, int nb_components, int Ah,
> >>                  v = s->v_scount[i];
> >>                  x = 0;
> >>                  y = 0;
> >> +                h_shift = c ? chroma_h_shift: 0;
> >> +                v_shift = c ? chroma_v_shift: 0;
> >>                  for (j = 0; j < n; j++) {
> >>                      block_offset = (((linesize[c] * (v * mb_y + y) * 8) +
> >>                                       (h * mb_x + x) * 8 * bytes_per_pixel) >> s->avctx->lowres);
> >>  
> >>                      if (s->interlaced && s->bottom_field)
> >>                          block_offset += linesize[c] >> 1;
> >> -                    if (   8*(h * mb_x + x) < s->width
> >> -                        && 8*(v * mb_y + y) < s->height) {
> >> +                    if (   8*(h * mb_x + x) < (s->width  + (1 << h_shift) - 1) >> h_shift
> >> +                        && 8*(v * mb_y + y) < (s->height + (1 << v_shift) - 1) >> v_shift) {
> > 
> > please move the w/h computation out of the block loop
> > it stays the same for a component and does not need to be
> > recalculated
> > theres a loop above that fills data[] that can probably be used to
> > fill w/h arrays
> 
> OK, but since there are only two possible values, I don't think filling
> arrays is necessary. Attached is an updated patch.

couldnt there be a alpha plane too ?

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Into a blind darkness they enter who follow after the Ignorance,
they as if into a greater darkness enter who devote themselves
to the Knowledge alone. -- Isha Upanishad
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20151205/edce9e6c/attachment.sig>