[FFmpeg-cvslog] h264: clear trailing bits in partially parsed NAL units

[Janne Grunau]

ffmpeg | branch: master | Janne Grunau <janne-libav at jannau.net> | Mon Dec 19 18:11:44 2011 +0100| [729ebb2f185244b0ff06d48edbbbbb02ceb4ed4e] | committer: Janne Grunau

h264: clear trailing bits in partially parsed NAL units

Trailing bits are likely to be non-zero if the NAL unit is truncated.
Clearing the bits make overreads of the bitstream less likely in this
case. Fixes playback of
http://streams.videolan.org/streams/mp4/Mr_MrsSmith-h264_aac.mp4 which
has a forbidden byte sequence of 0x00 0x00 0x00 in it SPS.

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=729ebb2f185244b0ff06d48edbbbbb02ceb4ed4e
---

 libavcodec/h264.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/libavcodec/h264.c b/libavcodec/h264.c
index 77acd71..a9a1051 100644
--- a/libavcodec/h264.c
+++ b/libavcodec/h264.c
@@ -3759,7 +3759,7 @@ static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size){
         int consumed;
         int dst_length;
         int bit_length;
-        const uint8_t *ptr;
+        uint8_t *ptr;
         int i, nalsize = 0;
         int err;
 
@@ -3809,6 +3809,9 @@ static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size){
         }
 
         if (h->is_avc && (nalsize != consumed) && nalsize){
+            // set trailing bits in the last partial byte to zero
+            if (bit_length & 7)
+                ptr[bit_length >> 3] = ptr[bit_length >> 3] & (0xff << 8 - (bit_length & 7));
             av_log(h->s.avctx, AV_LOG_DEBUG, "AVC: Consumed only %d bytes instead of %d\n", consumed, nalsize);
         }