[FFmpeg-devel] [PATCH] avcodec/h264, videotoolbox: do not return invalid frames on failure

wm4 nfxjfg at googlemail.com
Fri Nov 20 11:31:43 CET 2015


On Tue, 17 Nov 2015 15:19:29 +0100
wm4 <nfxjfg at gmail.com> wrote:

> If videotoolbox_common_end_frame failed, then the AVFrame was returned
> to the API user with the dummy buffer (in AVFrame.buf[0]) still set, and
> the decode call indicating success.
> 
> These "half-set" AVFrames with dummy buffer are a videotoolbox specific
> hack, because the decoder requires an allocated AVFrame for its internal
> logic. Videotoolbox on the other hand allocates its frame itself
> internally, and outputs it only on end_frame. At this point, the dummy
> buffer is replaced with the real frame (unless decoding fails).
> ---
> Better solutions welcome. For now, this fixes the API returning garbage
> which can crash or confuse API users,
> ---
>  libavcodec/h264_refs.c    | 3 ++-
>  libavcodec/videotoolbox.c | 2 ++
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/libavcodec/h264_refs.c b/libavcodec/h264_refs.c
> index 619f2ed..9d0641a 100644
> --- a/libavcodec/h264_refs.c
> +++ b/libavcodec/h264_refs.c
> @@ -515,7 +515,8 @@ void ff_h264_remove_all_refs(H264Context *h)
>  
>      if (h->short_ref_count && !h->last_pic_for_ec.f->data[0]) {
>          ff_h264_unref_picture(h, &h->last_pic_for_ec);
> -        ff_h264_ref_picture(h, &h->last_pic_for_ec, h->short_ref[0]);
> +        if (h->short_ref[0]->f->buf[0])
> +            ff_h264_ref_picture(h, &h->last_pic_for_ec, h->short_ref[0]);
>      }
>  
>      for (i = 0; i < h->short_ref_count; i++) {
> diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c
> index cc1e592..2f4d531 100644
> --- a/libavcodec/videotoolbox.c
> +++ b/libavcodec/videotoolbox.c
> @@ -353,6 +353,8 @@ static int videotoolbox_common_end_frame(AVCodecContext *avctx, AVFrame *frame)
>      AVVideotoolboxContext *videotoolbox = avctx->hwaccel_context;
>      VTContext *vtctx = avctx->internal->hwaccel_priv_data;
>  
> +    av_buffer_unref(&frame->buf[0]);
> +
>      if (!videotoolbox->session || !vtctx->bitstream)
>          return AVERROR_INVALIDDATA;
>  

Pushed.


More information about the ffmpeg-devel mailing list