[Ffmpeg-devel] h264: fix removing NAL unit's trailing zero bytes

[Michael Niedermayer]

Hi

On Mon, Apr 09, 2007 at 12:29:40PM +0200, Reinhard Nissl wrote:
> Hi,
> 
> the current code doesn't remove a trailing zero byte when dst_length is 1.
> 
> Consider the following byte sequence
> 
>     00 00 01 0a 00 00 00 01 09 ...
>                ^  ^
>                A  B
> 
> decode_nal() determines dst_length to be 1 (i. e. the byte between label
> A and B above). Though, this byte is a trailing zero byte as the spec
> says the the current NAL unit is terminated by a byte sequence 00 00 00.
> 
> Therefore the current code uses a loop to decrement dst_length
> accordingly. But the loop doesn't start as the loop condition checks for
> dst_length > 1, which should read dst_length > 0. The attached patch
> fixes this.
> 
> As a result of the fix, dst_length can now get 0 and therefore the
> calculation of bit_length may result in negative values, not to mention
> the access to ptr[-1] as argument to decode_rbsp_trailing().

decode_rbsp_trailing() shouldnt be called if dst_length==0


> 
> Regarding the access to ptr[-1]: it is save memory, i. e. the byte
> before label A in the sequence above.
> 
> Regarding bit_length to be negative: the next use of this variable is
> the call to init_get_bits() and there, a negative value is handled already.

patch looks ok

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No snowflake in an avalanche ever feels responsible. -- Voltaire
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20070409/77922a80/attachment.pgp>