[FFmpeg-devel] [PATCH] pgssubdec: only set w/h/linesize when allocating data

[Petri Hintukainen]

ti, 2016-11-22 kello 23:21 +0100, Andreas Cadhalpun kirjoitti:
> On 09.11.2016 23:27, Andreas Cadhalpun wrote:
> > 
> > Rects with positive w/h/linesize but no data are invalid.
> > 
> > Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
> > ---
> >  libavcodec/pgssubdec.c | 9 +++++----
> >  1 file changed, 5 insertions(+), 4 deletions(-)
> > 
> > diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
> > index cef477d..b50b37b 100644
> > --- a/libavcodec/pgssubdec.c
> > +++ b/libavcodec/pgssubdec.c
> > @@ -556,12 +556,13 @@ static int display_end_segment(AVCodecContext
> > *avctx, void *data,
> >  
> >          sub->rects[i]->x    = ctx->presentation.objects[i].x;
> >          sub->rects[i]->y    = ctx->presentation.objects[i].y;
> > -        sub->rects[i]->w    = object->w;
> > -        sub->rects[i]->h    = object->h;
> > -
> > -        sub->rects[i]->linesize[0] = object->w;
> >  
> >          if (object->rle) {
> > +            sub->rects[i]->w    = object->w;
> > +            sub->rects[i]->h    = object->h;
> > +
> > +            sub->rects[i]->linesize[0] = object->w;
> > +
> >              if (object->rle_remaining_len) {
> >                  av_log(avctx, AV_LOG_ERROR, "RLE data length %u is
> > %u bytes shorter than expected\n",
> >                         object->rle_data_len, object-
> > >rle_remaining_len);
> > 
> Ping. I intend to apply this in a few days.

Looks OK, w and h may contain old values if rle is NULL.

Maybe those could be set after decode_rle() result check, you could
then also simplify the error branch (no need to set w, h back to 0) ?

> Best regards,
> Andreas
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel