[FFmpeg-devel] [PATCH] avcodec/h264, videotoolbox: handle streams with multiple/changing PPS

[Aman Gupta]

From: Aman Gupta <aman at tmm1.net>

The videotoolbox hwaccel only receives SLICE and IDR_SLICE NALUs. This
works fine most of the time, but some streams fail to decode because
changes in PPS are not propagated to the VT decoder.

The failures in this case are incredibly annoying, as VTDecodeFrame()
still returns noErr. Simiarly the decoder callback is invoked with noErr,
and a NULL imageBuffer. Even though all the VT apis indicate success, no
frames are actually decoded.

When running ffmpeg via lldb however, some internal VT errors and
warnings show up all of a sudden. These suggest that the bitstream is
failing some internal consistency checks.

    $ ffmpeg -y -loglevel error -threads 1 -hwaccel videotoolbox \
             -i http://tmm1.s3.amazonaws.com/h264.ts -map v -f null /dev/null
    ...
    [h264 @ 0x7fdadc000000] hardware accelerator failed to decode picture
    Error while decoding stream #0:0: Unknown error occurred
    vt decoder cb: output image buffer is null
    ...

    $ lldb -- ffmpeg ...
    ...
    ffmpeg[49384:2009219] GVA error: AVC_RBSP::parseSliceHeader error
    ffmpeg[49384:2009219] GVA error: pushPicture parseSliceHeader
    vt decoder cb: output image buffer is null
    ffmpeg[49384:2009219] GVA warning: OutputQueueReadyCallback status = 1, buffer == 0x0
    [h264 @ 0x10300a200] hardware accelerator failed to decode picture: Unknown error occurred
    Error while decoding stream #0:0: Unknown error occurred
    ...

After this patch, there are no more errors and the sample decodes as
expected.
---
 libavcodec/h264dec.c      | 7 +++++++
 libavcodec/videotoolbox.c | 8 +++++---
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c
index 41c0964..af8b256 100644
--- a/libavcodec/h264dec.c
+++ b/libavcodec/h264dec.c
@@ -755,6 +755,13 @@ FF_ENABLE_DEPRECATION_WARNINGS
                                                        nal->size_bits);
             if (ret < 0 && (h->avctx->err_recognition & AV_EF_EXPLODE))
                 goto end;
+            if (avctx->hwaccel && avctx->hwaccel->pix_fmt == AV_PIX_FMT_VIDEOTOOLBOX) {
+                ret = avctx->hwaccel->decode_slice(avctx,
+                                                   nal->raw_data,
+                                                   nal->raw_size);
+                if (ret < 0)
+                    goto end;
+            }
             break;
         case H264_NAL_AUD:
         case H264_NAL_END_SEQUENCE:
diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c
index 1288aa5..1b5dffd 100644
--- a/libavcodec/videotoolbox.c
+++ b/libavcodec/videotoolbox.c
@@ -138,8 +138,6 @@ int ff_videotoolbox_h264_start_frame(AVCodecContext *avctx,
     VTContext *vtctx = avctx->internal->hwaccel_priv_data;
     H264Context *h  = avctx->priv_data;
 
-    vtctx->bitstream_size = 0;
-
     if (h->is_avc == 1) {
         return videotoolbox_buffer_copy(vtctx, buffer, size);
     }
@@ -373,8 +371,12 @@ static int videotoolbox_h264_end_frame(AVCodecContext *avctx)
 {
     H264Context *h = avctx->priv_data;
     AVFrame *frame = h->cur_pic_ptr->f;
+    VTContext *vtctx = avctx->internal->hwaccel_priv_data;
+    int ret;
 
-    return videotoolbox_common_end_frame(avctx, frame);
+    ret = videotoolbox_common_end_frame(avctx, frame);
+    vtctx->bitstream_size = 0;
+    return ret;
 }
 
 static int videotoolbox_mpeg_start_frame(AVCodecContext *avctx,
-- 
2.10.1 (Apple Git-78)