[FFmpeg-trac] #6989(avcodec:reopened): Hwaccel cuvid fails with “Error creating a NVDEC decoder: 1”

[FFmpeg]

#6989: Hwaccel cuvid fails with “Error creating a NVDEC decoder: 1”
-------------------------------------+-------------------------------------
             Reporter:  tkalliom     |                    Owner:
                 Type:  defect       |                   Status:  reopened
             Priority:  normal       |                Component:  avcodec
              Version:  git-master   |               Resolution:
             Keywords:  hwaccel      |               Blocked By:
  cuda cuvid NVDEC                   |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Changes (by DrocUf):

 * status:  closed => reopened
 * resolution:  invalid =>


Comment:

 Replying to [comment:1 philipl]:
 > That command line cannot work, so your results are not surprising.
 >
 > You either want to use the new nvdec hwaccel, or the old cuvid decoder.
 In either case, you need different arguments.
 >
 > To use the old style cuvid decoder:
 >
 >
 > {{{$ ffmpeg -hwaccel cuvid -c:v h264_cuvid small.mp4 small2.mp4}}}
 >
 >
 > To use the new style nvdec hwaccel:
 >
 > {{{$ ffmpeg -hwaccel nvdec small.mp4 small2.mp4}}}


 In the latest git snapshot both command lines will actually use exactly
 the same NVIDIA API. This is an actual bug in ffmpeg. The reason why the
 first one doesn't work is with `-hwaccel cuvid` ffmpeg will assign 0 to
 `CUVIDDECODECREATEINFO::ulNumDecodeSurfaces` and to
 `CUVIDDECODECREATEINFO::ulNumOutputSurfaces`. Therefore, a consequent call
 to `cuvidCreateDecoder` fails, because NVDEC doesn't have memory to store
 decoded pictures.

 Now follow me how it happens. When you specify `-hwaccel` in the command
 line parameters, ffmpeg assigns a constant from `enum HWAccelID` to
 `InputStream::hwaccel_id`. For `-hwaccel cuvid` this constant will be
 `HWACCEL_CUVID`. On other hand `-hwaccel nvdec` magically works not
 because it uses some secret sauce API, but because
 `InputStream::hwaccel_id` will be equal to `HWACCEL_GENERIC`. Here is how.
 First, `add_input_streams` will change `nvdec` to `cuda` here:

 {{{#!cpp
                 if (!strcmp(hwaccel, "nvdec"))
                     hwaccel = "cuda";
 }}}

 Next, it will compare it with the strings from a global variable
 `hwaccels` (defined in `ffmpeg_opt.c`) to derive `HWAccelID`. This table
 (surprise, surprise) has only three values: `"videotoolbox"`, `"qsv"`, and
 `"cuvid"`. I.e. it has neither `nvdec`, nor `cuda`. Yes, '''the only one
 correct command line argument for NVIDIA HW acceleration is `-hwaccel
 cuvid`'''. When you use `-hwaccel nvdec`, ffmpeg assigns `HWACCEL_GENERIC`
 to `InputStream::hwaccel_id`:

 {{{#!cpp
                     if (!ist->hwaccel_id) {
                         type = av_hwdevice_find_type_by_name(hwaccel);
                         if (type != AV_HWDEVICE_TYPE_NONE) {
                             ist->hwaccel_id = HWACCEL_GENERIC;
                             ist->hwaccel_device_type = type;
                         }
                     }
 }}}


 Here is what happens next. `get_format` is called. If
 `InputStream::hwaccel_id` is set to `HWACCEL_CUVID`, then `get_format`
 will call `cuvid_init`:

 {{{#!cpp
             ret = hwaccel->init(s); // <-- here is a call to cuvid_init
             if (ret < 0) {
                 av_log(NULL, AV_LOG_FATAL,
                        "%s hwaccel requested for input stream #%d:%d, "
                        "but cannot be initialized.\n", hwaccel->name,
                        ist->file_index, ist->st->index);
                 return AV_PIX_FMT_NONE;
             }
 }}}

 `cuvid_init` will allocate `InputStream::hw_frames_ctx`:

 {{{#!cpp
     ist->hw_frames_ctx = av_hwframe_ctx_alloc(hw_device_ctx);
 }}}

 Then `get_format` will assign a reference to this structure to
 `AVCodecContext::hw_frames_ctx`:

 {{{#!cpp
         if (ist->hw_frames_ctx) {
             s->hw_frames_ctx = av_buffer_ref(ist->hw_frames_ctx);
             if (!s->hw_frames_ctx)
                 return AV_PIX_FMT_NONE;
         }
 }}}

 Are you still following? Bear with me, I almost finished. Now the key
 moment where it all fails. When `ff_nvdec_decode_init` is called it
 executes the following lines:

 {{{#!cpp
     if (!avctx->hw_frames_ctx) {
         ret = ff_decode_get_hw_frames_ctx(avctx, AV_HWDEVICE_TYPE_CUDA);
         if (ret < 0)
             return ret;
     }
 }}}

 Remember that we initialized `hw_frames_ctx` in `cuvid_init`. Therefore in
 the code above `ff_decode_get_hw_frames_ctx` is not executed, an important
 consequence of this is that the following line in `ff_nvdec_frame_params`
 also is not executed:

 {{{#!cpp
     frames_ctx->initial_pool_size = dpb_size + 2;
 }}}

 `AVHWFramesContext::initial_pool_size` stays equal to zero. Then
 `ff_nvdec_decode_init` continues to here:

 {{{#!cpp
     params.ulNumDecodeSurfaces = frames_ctx->initial_pool_size;
     params.ulNumOutputSurfaces = frames_ctx->initial_pool_size;
 }}}

 Which Hey Presto assigns zero to
 `CUVIDDECODECREATEINFO::ulNumDecodeSurfaces` and to
 `CUVIDDECODECREATEINFO::ulNumOutputSurfaces`, and eventually
 `cuvidCreateDecoder` fails.

 Now why does it work with `-hwaccel nvdec`? It's funny, the reason is
 ''because it doesn't call `cuvid_init`''. Remember that this command line
 makes `InputStream::hwaccel_id` to be equal to `HWACCEL_GENERIC`? In this
 case ffmpeg omits `hwaccel->init(s)` call, then `hw_frames_ctx` is not
 allocated, then `ff_nvdec_decode_init` calls
 `ff_decode_get_hw_frames_ctx`, then `ff_nvdec_frame_params` assigns the
 correct value to `AVHWFramesContext::initial_pool_size`, then
 `ff_nvdec_decode_init` transfers it to
 `CUVIDDECODECREATEINFO::ulNumDecodeSurfaces` and to
 `CUVIDDECODECREATEINFO::ulNumOutputSurfaces`... finally
 `cuvidCreateDecoder` succeeds.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/6989#comment:7>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker