[FFmpeg-devel] h264.c avcC file reading error on PPS with trailing zeros, with tiny patch attached

[Francois Oligny-Lemieux]

Hi,

In .mp4 files are saved the SPS and PPS. When ffmpeg decodes the .mp4 video
extradata, it should allow trailing zeros at the end of those NAL units.

In original code, the SPS is decoded, then the PPS

        // Decode sps from avcC
        cnt = *(p+5) & 0x1f; // Number of sps
        p += 6;
        for (i = 0; i < cnt; i++) {
            nalsize = AV_RB16(p) + 2;
            if(decode_nal_units(h, p, nalsize) < 0) {
                av_log(avctx, AV_LOG_ERROR, "Decoding sps %d from avcC
failed\n", i);
                return -1;
            }
            p += nalsize;
        }
        // Decode pps from avcC
        cnt = *(p++); // Number of pps
        for (i = 0; i < cnt; i++) {
            nalsize = AV_RB16(p) + 2;
            if(decode_nal_units(h, p, nalsize)   != nalsize) {
                av_log(avctx, AV_LOG_ERROR, "Decoding pps %d from avcC
failed\n", i);
                return -1;
            }
            p += nalsize;
        }

The SPS part is OK because it just checks if decode_nal_units didn't fail.
The second part is different and less flexible because it requires that
decode_nal_unit consumes the same amount of bytes that were passed. I don't
know why both part were not coded the same. The problem is that this doesn't
pass when there are trailing zeros. So just making the second part's check
like the first one allows the decoding of .mp4 that have trailing zeros in
PPS (like the file I uploaded on
upload.mplayerhq.hu/MPlayer/incoming/HD4ED_au_720p60.03.mp4). Note
that QuickTime is able to
decode it, while FFmpeg fails; but with the patch, FFmpeg succeeds.

-            if(decode_nal_units(h, p, nalsize)  != nalsize) {
+            if(decode_nal_units(h, p, nalsize) < 0) {

Francois
-------------- next part --------------
A non-text attachment was scrubbed...
Name: h264.c.13aug2008.diff
Type: application/octet-stream
Size: 526 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080813/4c975a4f/attachment.obj>