[FFmpeg-devel] [PATCH v2] lavc/vvc: Remove assertions on qPy_{a, b}

[Nuo Mi]

On Thu, Aug 29, 2024 at 12:39 AM Frank Plowman <post at frankplowman.com>
wrote:

> On 28/08/2024 16:13, Anton Khirnov wrote:
> > Quoting Frank Plowman (2024-08-25 13:50:41)
> >> These assertions are not violated, even by illegal bitstreams.
> >
> > I don't follow this argument, not being violated by any reachable
> > runtime path is exactly how a correctly written assert should behave.
> >
>
> The point I was trying to make is that I have not seen the assertion
> violated (besides the false positives qPy_{a,b} = 63) in my fuzz
> testing.  Along with the fact the derivation procedure for qPy_{a,b} is
> not all that complicated and largely contained in this one function, one
> might consider the assertions unnecessary.
>
> That being said, I would prefer not to remove the assertions and instead
> to apply the first version of this patch.  It seems recent discussion on
> the thread of v1 is leaning back that way as well.
>
Hi Anton,
Frank's fix is right, we have a bug in the assertion itself.

Hi Frank,
Apologies for any noise I introduced during the review. I'll merge v1 if
there are no other objections

I introduced this issue because HEVC uses a magic number here:
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/hevc/filter.c#L115.
I mistakenly believed 52 was the MAX_QP for HEVC, so I used 63 in VVC.
After I merge v1, perhaps you could help by sending patches to replace the
magic numbers with MAX_QP.
Two patches may be needed:
The first would replace 51 and 52 in HEVC with MAX_QP and MAX_QP + 1.
The second would replace 63 and 64 in VVC.

Thank you


> --
> Frank
>