[FFmpeg-trac] #10096(undetermined:new): MediaCodec Encoder: Pixel ColorSpace of many Adreno GPUs is not supported

[FFmpeg]

#10096: MediaCodec Encoder: Pixel ColorSpace of many Adreno GPUs is not supported
-------------------------------------+-------------------------------------
             Reporter:  Ben T        |                    Owner:  (none)
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:
                                     |  undetermined
              Version:  unspecified  |               Resolution:
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Comment (by Ben T):

 Copy-Pasting response from Zhao Zhili in the mailing list where I
 originally incorrectly submitted this ticket:

 Thank you for your valuable test!

 There is an unexpected behavior in the test. The test was conducted with
 ffmpeg-kit, not ffmpeg cmdline.
 With FFmpeg cmdline and -hwcontext mediacodec, the encoder should have
 "-pix_fmt mediacodec" by
 default, and have an AVMediaCodecDeviceContext (with Android 8.0 or
 higher). The encoder doesn't
 setup properly with ffmpeg-kit.

 pix_fmt mediacodec should be supported by most of Android devices. It do
 have its limitation, e.g., it needs
 a Surface object or require Android 8.0, and it doesn't work with
 avfilter.

 The above issue can be fixed on the ffmpeg-kit's side. Now comes the dark
 corner of Mediacodec. It should
 work without pix_fmt mediacodec. Can you try specify the encoder pix_fmt
 as nv12, that's

 -hwaccel mediacodec -hwaccel_output_format mediacodec -i test_input.mp4
 -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12 -y test_output.mp4

 I guess these devices work with nv12.

 There is an old story "MediaCodec and device compatibility"

 https://urldefense.com/v3/__https://groups.google.com/g/android-
 platform/c/p_MoSk0JPNM__;!!N1d2nkBXpQ!K7sceiwL4SR1PIFjy4TY99mSRBvCQZi78fR4
 -CH4X5H9Q_KXbtmpW22WH9y-eagaazoZSyHLeITHcSseZZeyTDJ9$

 Ten years on, the situation doesn't improved much:
 1. There is an ugly API to get color format supported by a codec.
 2. It's only available in Java.
 3. Different codec component have different color format.

 Yes, we can use a bunch of JNI to get the supported color format. It must
 be done before create
 AVCodecContext. But different mediacodec component have different color
 format, user can select
 mediacodec by name (see the following patch), we have only one
 AVCodec.pix_fmts.

 avcodec/mediacodecenc: add option to select codec by name
 https://urldefense.com/v3/__https://patchwork.ffmpeg.org/project/ffmpeg/patch/tencent_729F52ABE34E4770972DB89AF840549A8208@qq.com/__;!!N1d2nkBXpQ!K7sceiwL4SR1PIFjy4TY99mSRBvCQZi78fR4
 -CH4X5H9Q_KXbtmpW22WH9y-eagaazoZSyHLeITHcSseZTsxLMxB$

 We can add query the supported color format and add some hints when
 mediacodec configure failed.
 Or we can add a helper function for user to query the color format. It
 will be a bunch of JNI code, limited
 by the ugly MediaCodecInfo.CodecCapabilities API. It's easy to be done
 with Java. TBH, it's much easier with
 trial and error, try nv12 then yuv420.

 We can do something with FFCodec.init_static_data, but we need a strategy
 first. I don't want to go with a long
 blacklist/whitelist.

 ** We only need one pixel format which supported by all devices,
 COLOR_FormatSurface doesn't count, and
 COLOR_FormatYUV420Flexible doesn't count. **

 Maybe Google can fix these issues with another 10 year.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10096#comment:1>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker