[FFmpeg-devel] [PATCH] Fix H.264 num_reorder_frames when not specified in VUI

[John Cox]

Hi

I was testing ffmpeg against my ITU derived test & conformance bitstreams and I
found ~20 failures were caused by the reordering code throwing away frames
until it has established the correct reordering level.  This can be fixed by
setting num_reorder_frames to MaxDpbSize before parsing the VUI.

This has the disadvantage that it will increase the latency through the decoder
but the advantage that frames are no longer lost (and it passes about 20 more
conformance tests).

I enclose a patch for this.

I used a modified tutorial01.c as the decoder for testing as ffmpeg itself
produced more failures (I'm guessing missing frames at the start or end - but
I'm not sure - I haven't investigated closely yet).  What options should I be
using to take an H.264 elementary bitstream in and produce a YUV file out with
no missing frames and no post processing?

Many thanks

John Cox

-------------- next part --------------
Index: libavcodec/h264.c
===================================================================
--- libavcodec/h264.c	(revision 17946)
+++ libavcodec/h264.c	(working copy)
@@ -104,6 +104,54 @@
     {0,2,0,2,7,10,7,10}
 };
 
+// Values of MaxDPB * 1024 from Table A-1
+static const int profile_dpb_size[42] =
+{
+    152064,  // 1
+    345600,
+    912384,
+    912384,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    912384,  // 2
+    1824768,
+    3110400,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    3110400,  // 3
+    6912000,
+    7864320,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    12582912,  // 4
+    12582912,
+    12582912,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    0,
+    42301440,  // 5
+    70778880,
+};
+
+
 #define LEVEL_TAB_BITS 8
 static int8_t cavlc_level_tab[7][1<<LEVEL_TAB_BITS][2];
 
@@ -7100,12 +7148,13 @@
     unsigned int sps_id;
     int i;
     SPS *sps;
+    int constraint_set3_flag;
 
     profile_idc= get_bits(&s->gb, 8);
     get_bits1(&s->gb);   //constraint_set0_flag
     get_bits1(&s->gb);   //constraint_set1_flag
     get_bits1(&s->gb);   //constraint_set2_flag
-    get_bits1(&s->gb);   //constraint_set3_flag
+    constraint_set3_flag = get_bits1(&s->gb);   //constraint_set3_flag
     get_bits(&s->gb, 4); // reserved
     level_idc= get_bits(&s->gb, 8);
     sps_id= get_ue_golomb_31(&s->gb);
@@ -7180,6 +7229,37 @@
     else
         sps->mb_aff= 0;
 
+    // Calculate best guess at reorder frames from profile DPB size
+    // If the DPB size in frames cannot be calculated because level_idc
+    // is an unknown value then num_reorder_frames is set to zero - this
+    // is safe enough as the actual reorder buffer size will be incremented
+    // whenever a frame is emitted out of order.
+    sps->num_reorder_frames = 0;
+    // 1st verify that we understand the level_idc on which we depend
+    if(sps->level_idc >= 10 && sps->level_idc <= 51) {
+        const int MaxDPBx1024 = profile_dpb_size[sps->level_idc - 10];
+        if(MaxDPBx1024 != 0) {
+            // Calculate MaxDpbSize from A.3.1 or A.3.2
+            int MaxDpbSize = MaxDPBx1024 /
+                (sps->mb_width * sps->mb_height * (2 - sps->frame_mbs_only_flag) * 384);
+            if(MaxDpbSize > 16)
+                MaxDpbSize = 16;
+
+            // Definition of num_reorder_frames in clause E.2.1
+            if(constraint_set3_flag
+               && (sps->profile_idc == 44 || sps->profile_idc == 86
+                   || sps->profile_idc == 100 || sps->profile_idc == 110
+                   || sps->profile_idc == 122 || sps->profile_idc == 244)) {
+                sps->num_reorder_frames = 0;
+            } else {
+                sps->num_reorder_frames = MaxDpbSize;
+            }
+
+            // Could set max_dec_frame_buffering here too but it isn't a
+            // parameter that is used anywhere (and so not in the sps structure)
+        }
+    }
+
     sps->direct_8x8_inference_flag= get_bits1(&s->gb);
 
 #ifndef ALLOW_INTERLACE
@@ -7812,9 +7892,11 @@
         //FIXME do something with unavailable reference frames
 
             /* Sort B-frames into display order */
-
-            if(h->sps.bitstream_restriction_flag
-               && s->avctx->has_b_frames < h->sps.num_reorder_frames){
+            /* sps.num_reorder_frames is set by the sps parsing code whether
+             * or not sps.bitstream_restriction_flag is set so can be used
+             * without checking 
+            */ 
+            if(s->avctx->has_b_frames < h->sps.num_reorder_frames){
                 s->avctx->has_b_frames = h->sps.num_reorder_frames;
                 s->low_delay = 0;
             }