[FFmpeg-devel] [PATCH V3] avcodec/h264_mp4toannexb_bsf: extract extradata for first coming frame

[Linjie Fu]

Add "new_nal_slice" to indicate the first coming nal_slice (including
H264_NAL_IDR_SLICE and H264_NAL_SLICE)

Extract extradata of streams when the first nal_slice comes, not only
H264_NAL_IDR_SLICE but also H264_NAL_SLICE.

This patch aims at the following issues:
1. the IDR frame is missing in the  first GOP of a stream
(common in live stream, IDR.No.Inband.SPPS.mkv in ticket #6418)
2. there is no IDR frame in the input stream.
(No.Inband.SPPS.No.IDR.mkv in ticket #6418)

Both clips could be decoded successfully using software decoding
but had problems in hwaccel using qsv before applying the patch.

V3: modified the commit message and the code duplication.

Signed-off-by: Linjie Fu <linjie.fu at intel.com>
---
 libavcodec/h264_mp4toannexb_bsf.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/libavcodec/h264_mp4toannexb_bsf.c b/libavcodec/h264_mp4toannexb_bsf.c
index fb3f24ea40..8dd9159b22 100644
--- a/libavcodec/h264_mp4toannexb_bsf.c
+++ b/libavcodec/h264_mp4toannexb_bsf.c
@@ -33,6 +33,7 @@ typedef struct H264BSFContext {
     int32_t  pps_offset;
     uint8_t  length_size;
     uint8_t  new_idr;
+    uint8_t  new_nal_slice;
     uint8_t  idr_sps_seen;
     uint8_t  idr_pps_seen;
     int      extradata_parsed;
@@ -236,13 +237,17 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx, AVPacket *out)
         if (!s->new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] & 0x80))
             s->new_idr = 1;
 
-        /* prepend only to the first type 5 NAL unit of an IDR picture, if no sps/pps are already present */
-        if (s->new_idr && unit_type == H264_NAL_IDR_SLICE && !s->idr_sps_seen && !s->idr_pps_seen) {
+        /* prepend to the first type 5 NAL unit of an IDR picture,
+         * or the first type 1 NAL unit of an IDR missing picture, 
+         * if no sps/pps are already present */
+        if (s->new_idr && !s->idr_sps_seen && !s->idr_pps_seen
+                    && (unit_type == H264_NAL_IDR_SLICE || (!s->new_nal_slice && H264_NAL_SLICE == unit_type))) {
             if ((ret=alloc_and_copy(out,
                                ctx->par_out->extradata, ctx->par_out->extradata_size,
                                buf, nal_size, 1)) < 0)
                 goto fail;
             s->new_idr = 0;
+            s->new_nal_slice = 1;
         /* if only SPS has been seen, also insert PPS */
         } else if (s->new_idr && unit_type == H264_NAL_IDR_SLICE && s->idr_sps_seen && !s->idr_pps_seen) {
             if (s->pps_offset == -1) {
-- 
2.17.1