[Libav-user] av_hwframe_transfer_data does not succeed from NV12 to YUV420P - How to get YUV420P from cuvid?
sw at jkqxz.net
Thu Jun 7 01:12:47 EEST 2018
On 06/06/18 08:46, Sergio Basurco wrote:> On 05/06/2018 22:23, Mark Thompson wrote:
>> On 05/06/18 11:15, Sergio Basurco wrote:
>>> I'm decoding h264 with ffmpeg. I want to use the hwaccel decoders. I'm using the cuvid decoder via API. In the fftools code there's a function "hwaccel_retrieve_data" that is supposed to convert the decoded frame (NV12) into any other format, I'm trying YUV420P.
>>> The conversion does not return any error, however the resulting data is not correct. Here's the original NV12 frame:
>>> And here's the conversion to YUV420P (only avframe->data and avframe->data have any data, data is expected to have the V data but it is missing).
>>> I think I'm on the right track, based on the code in hwcontext_cuda.c aparently YUV420P is a supported format, I cannot get my head around how I can tell the decoder to convert from NV12 to YUV420P though.
>>> Any tips will be appreciated, I'll update if I find anything.
>> See the documentation for av_hwframe_transfer_data() (<http://ffmpeg.org/doxygen/trunk/hwcontext_8h.html#abf1b1664b8239d953ae2cac8b643815a>), in particular:
>> "If src has an AVHWFramesContext attached, then the format of dst (if set) must use one of the formats returned by av_hwframe_transfer_get_formats(src, AV_HWFRAME_TRANSFER_DIRECTION_FROM). If dst has an AVHWFramesContext attached, then the format of src must use one of the formats returned by av_hwframe_transfer_get_formats(dst, AV_HWFRAME_TRANSFER_DIRECTION_TO)"
>> For CUDA, no conversion during transfer is supported so the only usable output format returned by av_hwframe_transfer_get_formats() is the same format as the GPU-side frame itself (NV12 in your case): <http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/hwcontext_cuda.c#l176>.
> Hi Mark,
> I did misread that bit of documentation, thanks!
> I'm trying to understand the supported_formats array found in hwcontext_cuda.c <https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/hwcontext_cuda.c#L35>. Can the hw context be initialized so that the decoder returns YUV420P instead of NV12?
The code there is saying that YUV420P is supported as a format for CUDA frames, but that doesn't mean that it is also supported for decoding to - what formats are supported there is dependent on the decoder hardware.
Looking at the decoder code it looks like it only supports NV12 for 8-bit decoding, so YUV420P in CUDA therefore can't be used there (on the other hand, it can be used for encoding and some filter operations).
More information about the Libav-user