[FFmpeg-devel] [PATCH] avcodec/vc1: fix overlap smoothing filter for P frames

[Carl Eugen Hoyos]

2018-05-29 22:02 GMT+02:00, Jerome Borsboom <jerome.borsboom at carpalis.nl>:
> The v_overlap_filter needs to run on the colocated block of the previous
> macroblock. For the luma plane, the colocated block is located two blocks
> on the left instead of one.
>
> Signed-off-by: Jerome Borsboom <jerome.borsboom at carpalis.nl>
> ---
> This should fix the issue with the SA10100.vc1 test file.
>
>  libavcodec/vc1_loopfilter.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/vc1_loopfilter.c b/libavcodec/vc1_loopfilter.c
> index 4c0de7c025..676922aa18 100644
> --- a/libavcodec/vc1_loopfilter.c
> +++ b/libavcodec/vc1_loopfilter.c
> @@ -145,8 +145,8 @@ void ff_vc1_p_overlap_filter(VC1Context *v)
>
>      if (v->fcm != ILACE_FRAME)
>          for (i = 0; i < block_count; i++) {
> -            if (s->mb_x && v->mb_type[0][s->block_index[i] - 1] &&
> -                (s->first_slice_line || v->mb_type[0][s->block_index[i] -
> s->block_wrap[i] - 1]))
> +            if (s->mb_x && v->mb_type[0][s->block_index[i] - 2 + (i > 3)]
> &&
> +                (s->first_slice_line || v->mb_type[0][s->block_index[i] -
> s->block_wrap[i] - 2 + (i > 3)]))

This also fixes the frame number 5 and 6 of SSL0013.rcv (ticket #7171),
the last two frames are still incorrect.

Thank you, Carl Eugen