[FFmpeg-trac] #9951(undetermined:new): OpenEncodeSessionEx failed: out of memory (10): (no details)

[FFmpeg]

#9951: OpenEncodeSessionEx failed: out of memory (10): (no details)
-------------------------------------+-------------------------------------
             Reporter:  javex        |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:
  undetermined                       |  unspecified
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug: I'm trying to use my Nvidia 1660 Super to transcode a
 video (see info below) using the latest code from the master branch
 (a1bfb5290e) and I'm getting the following error:

 {{{
 [h264_nvenc @ 0x55d14231ab00] OpenEncodeSessionEx failed: out of memory
 (10): (no details)
 [h264_nvenc @ 0x55d14231ab00] Nvenc unloaded
 Error initializing output stream 0:0 -- Error while opening encoder for
 output stream #0:0 - maybe incorrect parameters such as bit_rate, rate,
 width or height
 }}}

 I've attached a full debug log (stderr output) in case it helps.

 Info about the system:
 {{{
 $ uname -a
 Linux media-server.home.xevaj.eu 5.15.71-1-lts #1 SMP Wed, 28 Sep 2022
 10:41:31 +0000 x86_64 GNU/Linux
 $ nvidia-smi
 Fri Sep 30 22:25:27 2022
 +-----------------------------------------------------------------------------+
 | NVIDIA-SMI 515.76       Driver Version: 515.76       CUDA Version: 11.7
 |
 |-------------------------------+----------------------+----------------------+
 | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr.
 ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute
 M. |
 |                               |                      |               MIG
 M. |
 |===============================+======================+======================|
 |   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |
 N/A |
 |  0%   61C    P0    34W / 125W |      0MiB /  6144MiB |      0%
 Default |
 |                               |                      |
 N/A |
 +-------------------------------+----------------------+----------------------+

 +-----------------------------------------------------------------------------+
 | Processes:
 |
 |  GPU   GI   CI        PID   Type   Process name                  GPU
 Memory |
 |        ID   ID                                                   Usage
 |
 |=============================================================================|
 |  No running processes found
 |
 +-----------------------------------------------------------------------------+

 }}}

 Not sure what else I can supply, I don't think the video file is the
 culprit, but happy to debug with others if you have some good samples I
 could use to confirm.

 How to reproduce:
 {{{
 % ./ffmpeg -v 9 -loglevel 99 -hwaccel cuda -hwaccel_output_format cuda  -i
 ffmpeg-samples/input.mkv -c:v h264_nvenc ffmpeg-samples/output.mkv
 ffmpeg version
 built on
 ./ffmpeg --version
 ffmpeg version N-108445-ga1bfb5290e Copyright (c) 2000-2022 the FFmpeg
 developers
   built with gcc 12.2.0 (GCC)
   configuration: --samples=ffmpeg-samples --cc= --enable-cuda-llvm
 --enable-nvenc
   libavutil      57. 38.100 / 57. 38.100
   libavcodec     59. 48.100 / 59. 48.100
   libavformat    59. 33.100 / 59. 33.100
   libavdevice    59.  8.101 / 59.  8.101
   libavfilter     8. 49.100 /  8. 49.100
   libswscale      6.  8.112 /  6.  8.112
   libswresample   4.  9.100 /  4.  9.100
 }}}

 Video data:
 {{{
 ./ffmpeg -i ffmpeg-samples/input.mkv
 ffmpeg version N-108445-ga1bfb5290e Copyright (c) 2000-2022 the FFmpeg
 developers
   built with gcc 12.2.0 (GCC)
   configuration: --samples=ffmpeg-samples --cc= --enable-cuda-llvm
 --enable-nvenc
   libavutil      57. 38.100 / 57. 38.100
   libavcodec     59. 48.100 / 59. 48.100
   libavformat    59. 33.100 / 59. 33.100
   libavdevice    59.  8.101 / 59.  8.101
   libavfilter     8. 49.100 /  8. 49.100
   libswscale      6.  8.112 /  6.  8.112
   libswresample   4.  9.100 /  4.  9.100
 Input #0, matroska,webm, from 'ffmpeg-samples/input.mkv':
   Metadata:
     encoder         : libebml v1.3.6 + libmatroska v1.4.9
     creation_time   : 2018-07-11T04:28:24.000000Z
   Duration: 00:42:20.71, start: 0.000000, bitrate: 9310 kb/s
   Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 1920x1080 [SAR 1:1
 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn (default)
     Metadata:
       BPS-eng         : 8668560
       DURATION-eng    : 00:42:20.705000000
       NUMBER_OF_FRAMES-eng: 60916
       NUMBER_OF_BYTES-eng: 2753031717
       _STATISTICS_WRITING_APP-eng: mkvmerge v24.0.0 ('Beyond The Pale')
 64-bit
       _STATISTICS_WRITING_DATE_UTC-eng: 2018-07-11 04:28:24
       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
   Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s
 (default)
     Metadata:
       BPS-eng         : 640000
       DURATION-eng    : 00:42:20.672000000
       NUMBER_OF_FRAMES-eng: 79396
       NUMBER_OF_BYTES-eng: 203253760
       _STATISTICS_WRITING_APP-eng: mkvmerge v24.0.0 ('Beyond The Pale')
 64-bit
       _STATISTICS_WRITING_DATE_UTC-eng: 2018-07-11 04:28:24
       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
   Stream #0:2(eng): Subtitle: subrip
     Metadata:
       BPS-eng         : 99
       DURATION-eng    : 00:41:29.453000000
       NUMBER_OF_FRAMES-eng: 923
       NUMBER_OF_BYTES-eng: 30928
       _STATISTICS_WRITING_APP-eng: mkvmerge v24.0.0 ('Beyond The Pale')
 64-bit
       _STATISTICS_WRITING_DATE_UTC-eng: 2018-07-11 04:28:24
       _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
 }}}
 Patches should be submitted to the ffmpeg-devel mailing list and not this
 bug tracker.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9951>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker