[FFmpeg-devel] [PATCH] avcodec/mediacodec_sw_buffer: Fix segmentation fault with decoding on android oreo (corrected)

[Aman Gupta]

On Thu, Jul 19, 2018 at 10:08 AM Peter Bennett <pb.mythtv at gmail.com> wrote:

> This is a correction of the earlier submission of this patch.
>
> avcodec_receive_frame consistently causes a seg fault when decoding 1080i
> mpeg2
> on android version oreo. When copying the frame, the second plane in the
> buffer
> follows on immediately after 1080 lines of the first plane, but the code
> assumes
> it is after 1088 lines of the first plane, based on slice_height. It
> crashes on
> copying data for the second plane when it hits the actual end of the data
> and
> starts accessing addresses beyond that.
>
> Instead of using slice_height here, change to use use height. slice_height
> is
> used at other places in this module and I do not know if they also need to
> be
> changed. I have confirmed that with this change, decoding works correctly
> on android oreo as well as on the prior version, android nougat.
>

This issue is not specific to 1080i mpeg2, and can be reproduced with 1080p
h264 as well.

slice_height is definitely the correct variable to use here. The problem is
it is not being set correctly. I will send an updated patch.

Aman


> ---
>  libavcodec/mediacodec_sw_buffer.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/mediacodec_sw_buffer.c
> b/libavcodec/mediacodec_sw_buffer.c
> index 92428e85f0..30a53f05b3 100644
> --- a/libavcodec/mediacodec_sw_buffer.c
> +++ b/libavcodec/mediacodec_sw_buffer.c
> @@ -150,7 +150,7 @@ void
> ff_mediacodec_sw_buffer_copy_yuv420_semi_planar(AVCodecContext *avctx,
>          } else if (i == 1) {
>              height = avctx->height / 2;
>
> -            src += s->slice_height * s->stride;
> +            src += s->height * s->stride;
>              src += s->crop_top * s->stride;
>              src += s->crop_left;
>          }
> --
> 2.17.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>