[FFmpeg-devel] [PATCH] lavc/cuvid: fail early if GPU can't handle given video parameters

[Pavel Koshevoy]

On Mon, Jan 9, 2017 at 3:12 AM, wm4 <nfxjfg at googlemail.com> wrote:
> On Mon,  2 Jan 2017 14:26:45 -0700
> pkoshevoy at gmail.com wrote:
>
>> From: Pavel Koshevoy <pkoshevoy at gmail.com>

>> +    ret = convert_to_cuda_video_chroma_format(avctx->pix_fmt, &probed_chroma_format);
>> +    if (ret < 0) {
>> +        // pixel format is not supported:
>> +        return ret;
>> +    }
>> +    probed_width = avctx->coded_width ? avctx->coded_width : 1280;
>> +    probed_height = avctx->coded_height ? avctx->coded_height : 720;
>
> IMO relying on these fields is a bit of a problem, since they don't
> even need to be set (the h264 bitstream can contain them).
>
> It's especially bad for pix_fmt, since my application for example
> doesn't set this field at all in many cases.

I've posted a new version of the patch which assumes
AV_PIX_FMT_YUV420P if avctx->pix_fmt is unset.  This version of the
patch should preserve the existing behavior for those use cases where
pix_fmt is AV_PIX_FMT_NONE, width = 0, height = 0...  although it
won't help if those fields are un-initialized a contain random values.

The real problem I am trying to solve -- I need to find the "best"
decoder for the incoming stream, or file.  There are multiple decoders
available for H264 ... Selecting based on supported output pixel
formats would eliminate some.  Selecting based on AVCodecParameters
would allow to eliminate those decoders that can't handle given
parameters, although that appears to be insufficient.  Perhaps
selecting based on given codec extradata (PPS/SPS) would be more
robust.  extradata is usually part of the bitstream... is there an API
to feed extradata to the decoder to see if it likes it?

Thank you,
    Pavel