[FFmpeg-devel] [PATCH] Fix crash if av_vdpau_bind_context() is not used.

[wm4]

On Mon, 9 Oct 2017 03:04:53 +0300
Ivan Kalvachev <ikalvachev at gmail.com> wrote:

> The public functions av_alloc_vdpaucontext() and
> av_vdpau_alloc_context() are allocating AVVDPAUContext
> structure that is supposed to be placed in avctx->hwaccel_context.
> 
> However the rest of libavcodec/vdpau.c uses avctx->hwaccel_context
> as struct VDPAUHWContext, that is bigger and does contain
> AVVDPAUContext as first member.
> 
> The usage includes write to the new variables in the bigger stuct,
> without checking for block size.
> 
> Fix by always allocating the bigger structure.
> 
> BTW,
> I have no idea why the new fields haven't simply been added to the
> existing struct...
> It seems that the programmer who wrote this has been aware of the problem,
> because av_vdpau_bind_context reallocates the structure.
> 
> It might be good idea to check the other usages of this reallocation function.
> 
> Best Regards
>    Ivan Kalvachev

IMO not really worth fixing at this point, because this is the old-old
vdpau API. Even av_vdpau_bind_context() (which does not require using
av_alloc_vdpaucontext()) is deprecated. Or rather should be - I just
haven't bothered deprecating it because the deprecation dance is too
messy. In any case, you shouldn't use any of those APIs - use the
generic hwaccel API instead (setting hw_frames_ctx or hw_device_ctx).