[FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

[Nuo Mi]

On Wed, Feb 21, 2024 at 7:10 PM J. Dekker <jdek at itanimul.li> wrote:

> Over/underflow in some cases.
>
> Signed-off-by: J. Dekker <jdek at itanimul.li>
> ---
>  libavcodec/x86/hevcdsp_init.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c
> index 31e81eb11f..11cb1b3bfd 100644
> --- a/libavcodec/x86/hevcdsp_init.c
> +++ b/libavcodec/x86/hevcdsp_init.c
> @@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const
> int bit_depth)
>          if (EXTERNAL_SSE2(cpu_flags)) {
>              c->hevc_v_loop_filter_chroma =
> ff_hevc_v_loop_filter_chroma_12_sse2;
>              c->hevc_h_loop_filter_chroma =
> ff_hevc_h_loop_filter_chroma_12_sse2;
> -            if (ARCH_X86_64) {
> -                c->hevc_v_loop_filter_luma =
> ff_hevc_v_loop_filter_luma_12_sse2;
> -                c->hevc_h_loop_filter_luma =
> ff_hevc_h_loop_filter_luma_12_sse2;
> -            }
> +            // FIXME: 12-bit luma deblock over/underflows in some cases
> +            // if (ARCH_X86_64) {
> +            //     c->hevc_v_loop_filter_luma =
> ff_hevc_v_loop_filter_luma_12_sse2;
> +            //     c->hevc_h_loop_filter_luma =
> ff_hevc_h_loop_filter_luma_12_sse2;
> +            // }
>              SAO_BAND_INIT(12, sse2);
>              SAO_EDGE_INIT(12, sse2);
>
Hi Dekker,
VVC will utilize this function as well.
Could you please share the HEVC clip or data that caused the overflow?
We'll make efforts to address it during the VVC porting

Thank you.

>
> --
> 2.43.2
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>