[FFmpeg-trac] #9603(avutil:new): Heap corruption using d3d11va hwaccel without pool

[FFmpeg]

#9603: Heap corruption using d3d11va hwaccel without pool
---------------------------------+--------------------------------------
             Reporter:  Adion    |                     Type:  defect
               Status:  new      |                 Priority:  normal
            Component:  avutil   |                  Version:  git-master
             Keywords:  d3d11va  |               Blocked By:
             Blocking:           |  Reproduced by developer:  0
Analyzed by developer:  0        |
---------------------------------+--------------------------------------
 Summary of the bug:
 d3d11va_alloc_single() which is used when no pool is used makes a call to
 wrap_texture_buf()
 This in turn writes to frames_hwctx->texture_infos, but this would not be
 initialized if no pool is used, causing heap corruption and eventually a
 crash.

 Possible solution:
 Adding check to wrap_texture_buf seems to work fine to only update
 texture_infos if it's safe.
 if (s->nb_surfaces_used<s->nb_surfaces)

 Additional error logging/check might be needed in case pool is used
 (s->nb_surfaces>0) but I did not check this use case.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9603>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker