[FFmpeg-devel] [PATCH] avcodec/mediacodecenc: Extract configOBUs from AV1CodecConfigurationRecord

[Zhao Zhili]

> On Oct 5, 2024, at 05:01, Martin Storsjö <martin at martin.st> wrote:
> 
> On Sat, 5 Oct 2024, Zhao Zhili wrote:
> 
>> From: Zhao Zhili <zhilizhao at tencent.com>
>> 
>> MediaCodec can generate AV1CodecConfigurationRecord, which shouldn't
>> be put into packet->data. Skip four bytes and extract configOBUs
>> if it exist.
>> ---
>> I did some test on Pixel 8 Pro. AV1 hardware encoding works with a lot
>> of bugs:
>> 
>> 1. It's broken for width non-aligned to 16. For width 1080 and pixel
>> format YUV420P, MediaCodec use 1080 as stride. For pixel format NV12,
>> MediaCodec use 1088 as stride. There is no API to get the stride info.
>> AMEDIAFORMAT_KEY_STRIDE doesn't work. And set stride to MediaCodec has
>> no effect from my test, at least on that device. We know the buffer
>> size provided by MediaCodec, but we still cannot get stride by
>> buf_size / height:
>> 
>> 1) For YUV420P, buf_size = 1080 * height
>> 2) For NV12, buf_size = 1080 + 1088 * (height - 1). Yes, buf_size doesn't
>> count last line's padding :(
> 
> Isn't this pretty much the case for the encoders for other codecs as well - there aren't really any compat guarantees for how they behave for widths that aren't a multiple of 16? At least back when there when Android added CTS tests to guarantee some sort of cross device consistent behaviour, they only tested/mandated the behviour for a couple resolutions, that all were even multiples of 16.

> 
> I guess the difference here is whether it's possible to do cropping in the same way as via the h264_metadata/hevc_metadata BSFs?

All codecs have the same issue for dst buffer stride. Yes we can workaround it for H.264/H.265 with
h264_metadata/hevc_metadata BSFs. I think we should do the same for AV1.

Even without dst buffer stride issue, some devices still have align requirement on video size. For
example, when send frame to encoder via Surface/ANativeWindow, there is no explicit memcpy and
OS handles alignment of the underlying buffer. A 1080x1920 video turn into 1072x1920 after encoding
with MediaTek device (not with FFmpeg MediaCodec wrapper but call MediaCodec API directly in App).
Take care of alignment/crop within our wrapper rather than leave it to OS implementation resolved these
mess.

> 
> // Martin
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".