[Ffmpeg-devel] [RFC] Fraps v2 support

[Michael Niedermayer]

Hi

On Sun, Nov 05, 2006 at 06:55:35AM +0200, Kostya wrote:
> On Sat, Nov 04, 2006 at 11:57:15AM +0100, Michael Niedermayer wrote:
> > Hi
> > 
> > On Sat, Nov 04, 2006 at 08:04:12AM +0200, Kostya wrote:
> > [...]
> > > > > +        }
> > > > 
> > > > there is a possible problem here, and that is that the sum of the 2 counts
> > > > can overflow, if it does then this loop will reorder too many elements 
> > > > and by doing that trash the n0/n1 pointers which then a few lines later
> > > > could lead to a stack overflow due the recursive vlc building being
> > > > run on a tree which isnt one as it contains a loop from the corruption
> > > 
> > > Added an explicit check for that
> > 
> > well, as the counts are added recursivly, the check is insufficient to
> > prevent overflow and corrupting the pointer structure, though i dont know
> > if it can lead to loops (i failed to find an example where it did)
> > also keep in mind that the SWAP check is signed currently not unsigned
> > if it where unsigned the overflows would happen a few iterations later ...
> 
> a) sum of symbol frequencies should be <= width * height

ideally ...
but not with intentioanlly created files to crash your code


> b) no matter if we add them recursively, the count of huffman tree root will
> be equal to the sum of counts of both its childs or counts of all leaves.

agree


> Thus, is the sum does not overflow (check is done with int64_t sum, so when
> when something will appear in bits > 31 it will be catched), all nodes' counts
> are <= sum so overflow won't happen.

i indeed have misread your code somehow ... sorry

but your code still doesnt look safe, as you add signed int32, so large and
small counts can cancell each other out and then intermediats in the final
calculation can overflow

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or copy any book
Today you'd get arrested for mere telling someone where the library is