[RFC] H.264/SQV3 separation: h264data.h
Next up on the list is moving the tables in h264data.h to a .c file that can be compiled and shared by the H.264 and SVQ3 decoder. This requires adding an ff_ prefix to the names of the tables, removing the static keyword and then moving h264data.h to h264data.c while replacing the contents of h264data.h with appropriate extern declarations. I have done this in my local tree, but the patch is 174kB big (and terribly boring), so I'm not sending it here. OK to proceed (in several steps)? Will this need to be benchmarked at some point? Diego
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote:
Next up on the list is moving the tables in h264data.h to a .c file that can be compiled and shared by the H.264 and SVQ3 decoder. This requires adding an ff_ prefix to the names of the tables, removing the static keyword and then moving h264data.h to h264data.c while replacing the contents of h264data.h with appropriate extern declarations.
I have done this in my local tree, but the patch is 174kB big (and terribly boring), so I'm not sending it here. OK to proceed (in several steps)?
yes
Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Many that live deserve death. And some that die deserve life. Can you give it to them? Then do not be too eager to deal out death in judgement. For even the very wise cannot see all ends. -- Gandalf
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote:
Next up on the list is moving the tables in h264data.h to a .c file that can be compiled and shared by the H.264 and SVQ3 decoder. This requires adding an ff_ prefix to the names of the tables, removing the static keyword and then moving h264data.h to h264data.c while replacing the contents of h264data.h with appropriate extern declarations.
I have done this in my local tree, but the patch is 174kB big (and terribly boring), so I'm not sending it here. OK to proceed (in several steps)?
yes
Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit
Hmmmmm I ran the following benchmarks on my K6-III with two different samples: http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg I used the following MPlayer command lines to benchmark: mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null I made five runs for each sample and vo and threw away the best and worst times. The results are attached. Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive. Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?) Are these intermediate results worth anything at all or should the benchmarks be done once the split is complete to be significant? I remember that WMV2 decoding times improved after the split from msmpeg4. Please point me in the right direction. Diego
On Fri, Nov 28, 2008 at 09:46:19PM +0100, Diego Biurrun wrote:
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote:
Next up on the list is moving the tables in h264data.h to a .c file that can be compiled and shared by the H.264 and SVQ3 decoder. This requires adding an ff_ prefix to the names of the tables, removing the static keyword and then moving h264data.h to h264data.c while replacing the contents of h264data.h with appropriate extern declarations.
I have done this in my local tree, but the patch is 174kB big (and terribly boring), so I'm not sending it here. OK to proceed (in several steps)?
yes
Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit
Hmmmmm
I ran the following benchmarks on my K6-III with two different samples:
http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg
I used the following MPlayer command lines to benchmark:
mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null
I made five runs for each sample and vo and threw away the best and worst times. The results are attached.
Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive.
Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?)
time ffmpeg ... >&/dev/null
Are these intermediate results worth anything at all or should the benchmarks be done once the split is complete to be significant?
I remember that WMV2 decoding times improved after the split from msmpeg4. Please point me in the right direction.
iam not sure, one thing that may or may not help is figuring out which table it is that causes the slowdown. I dont really belive it is a specific table, rather gcc doing something randomly silly. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Republics decline into democracies and democracies degenerate into despotisms. -- Aristotle
On Sat, Nov 29, 2008 at 05:28:27PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 09:46:19PM +0100, Diego Biurrun wrote:
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote:
Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit
Hmmmmm
I ran the following benchmarks on my K6-III with two different samples:
http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg
I used the following MPlayer command lines to benchmark:
mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null
I made five runs for each sample and vo and threw away the best and worst times. The results are attached.
Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive.
Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?)
time ffmpeg ... >&/dev/null
Sure, but what options would I use?
Are these intermediate results worth anything at all or should the benchmarks be done once the split is complete to be significant?
I remember that WMV2 decoding times improved after the split from msmpeg4. Please point me in the right direction.
iam not sure, one thing that may or may not help is figuring out which table it is that causes the slowdown. I dont really belive it is a specific table, rather gcc doing something randomly silly.
I ran the test on my K6-III with gcc 4.1, but I could reproduce similar numbers on my G4 with gcc 4.3. So all in all this might actually be for real. I redid the benchmarks with the ugly but complete split I have in my local tree: http://www1.mplayerhq.hu/~diego/split.diff The numbers are attached. It still appears to be slower, but not that much. Since intermediate numbers do not appear to paint too accurate a picture my suggestion is to go ahead with the split and start comparing when it is finished. Michael? Diego
On Mon, Dec 01, 2008 at 11:25:59AM +0100, Diego Biurrun wrote:
On Sat, Nov 29, 2008 at 05:28:27PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 09:46:19PM +0100, Diego Biurrun wrote:
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote:
Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit
Hmmmmm
I ran the following benchmarks on my K6-III with two different samples:
http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg
I used the following MPlayer command lines to benchmark:
mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null
I made five runs for each sample and vo and threw away the best and worst times. The results are attached.
Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive.
Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?)
time ffmpeg ... >&/dev/null
Sure, but what options would I use?
Are these intermediate results worth anything at all or should the benchmarks be done once the split is complete to be significant?
I remember that WMV2 decoding times improved after the split from msmpeg4. Please point me in the right direction.
iam not sure, one thing that may or may not help is figuring out which table it is that causes the slowdown. I dont really belive it is a specific table, rather gcc doing something randomly silly.
I ran the test on my K6-III with gcc 4.1, but I could reproduce similar numbers on my G4 with gcc 4.3. So all in all this might actually be for real.
I redid the benchmarks with the ugly but complete split I have in my local tree:
http://www1.mplayerhq.hu/~diego/split.diff
The numbers are attached. It still appears to be slower, but not that much.
Since intermediate numbers do not appear to paint too accurate a picture my suggestion is to go ahead with the split and start comparing when it is finished. Michael?
.. and here are the numbers I forgot to attach as usual .. Diego
On Mon, Dec 01, 2008 at 11:25:59AM +0100, Diego Biurrun wrote:
On Sat, Nov 29, 2008 at 05:28:27PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 09:46:19PM +0100, Diego Biurrun wrote:
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote:
Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit
Hmmmmm
I ran the following benchmarks on my K6-III with two different samples:
http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg
I used the following MPlayer command lines to benchmark:
mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null
I made five runs for each sample and vo and threw away the best and worst times. The results are attached.
Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive.
Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?)
time ffmpeg ... >&/dev/null
Sure, but what options would I use?
maybe -i your_file -y -an -f null /dev/null [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The educated differ from the uneducated as much as the living from the dead. -- Aristotle
On Mon, Dec 01, 2008 at 02:02:57PM +0100, Michael Niedermayer wrote:
On Mon, Dec 01, 2008 at 11:25:59AM +0100, Diego Biurrun wrote:
On Sat, Nov 29, 2008 at 05:28:27PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 09:46:19PM +0100, Diego Biurrun wrote:
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote:
Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit
Hmmmmm
I ran the following benchmarks on my K6-III with two different samples:
http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg
I used the following MPlayer command lines to benchmark:
mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null
I made five runs for each sample and vo and threw away the best and worst times. The results are attached.
Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive.
Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?)
time ffmpeg ... >&/dev/null
Sure, but what options would I use?
maybe -i your_file -y -an -f null /dev/null
OK, I redid the benchmarks directly with ffmpeg on my K6-III, here are the results of 5 runs each, highest and lowest result discarded, all services stopped: ffmpeg-split -i Aladin.mpg -an -f null -y -benchmark -v -1 /dev/null bench: utime=24.446s bench: utime=24.466s bench: utime=24.530s ffmpeg-vanilla -i Aladin.mpg -an -f null -y -benchmark -v -1 /dev/null bench: utime=24.502s bench: utime=24.510s bench: utime=24.490s ffmpeg-split -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=284.858s bench: utime=284.110s bench: utime=283.610s ffmpeg-vanilla -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=283.438s bench: utime=282.830s bench: utime=283.170s The result seems to be no slowdown or a very minor slowdown below 1%. So can I commit the next part of the split and continue working on it? Diego
On Sun, Dec 14, 2008 at 04:24:43PM +0100, Diego Biurrun wrote:
On Mon, Dec 01, 2008 at 02:02:57PM +0100, Michael Niedermayer wrote:
On Mon, Dec 01, 2008 at 11:25:59AM +0100, Diego Biurrun wrote:
On Sat, Nov 29, 2008 at 05:28:27PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 09:46:19PM +0100, Diego Biurrun wrote:
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote: > Will this need to be benchmarked at some point?
id say it cant hurt to benchmark it before commit
Hmmmmm
I ran the following benchmarks on my K6-III with two different samples:
http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg
I used the following MPlayer command lines to benchmark:
mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null
I made five runs for each sample and vo and threw away the best and worst times. The results are attached.
Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive.
Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?)
time ffmpeg ... >&/dev/null
Sure, but what options would I use?
maybe -i your_file -y -an -f null /dev/null
OK, I redid the benchmarks directly with ffmpeg on my K6-III, here are the results of 5 runs each, highest and lowest result discarded, all services stopped: [...] ffmpeg-split -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=284.858s bench: utime=284.110s bench: utime=283.610s
ffmpeg-vanilla -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=283.438s bench: utime=282.830s bench: utime=283.170s
The result seems to be no slowdown or a very minor slowdown below 1%.
So can I commit the next part of the split and continue working on it?
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Everything should be made as simple as possible, but not simpler. -- Albert Einstein
On Sun, Dec 14, 2008 at 05:03:04PM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 04:24:43PM +0100, Diego Biurrun wrote:
On Mon, Dec 01, 2008 at 02:02:57PM +0100, Michael Niedermayer wrote:
maybe -i your_file -y -an -f null /dev/null
OK, I redid the benchmarks directly with ffmpeg on my K6-III, here are the results of 5 runs each, highest and lowest result discarded, all services stopped: [...] ffmpeg-split -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=284.858s bench: utime=284.110s bench: utime=283.610s
ffmpeg-vanilla -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=283.438s bench: utime=282.830s bench: utime=283.170s
The result seems to be no slowdown or a very minor slowdown below 1%.
So can I commit the next part of the split and continue working on it?
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
What is your suggestion? Diego
On Sun, Dec 14, 2008 at 05:09:47PM +0100, Diego Biurrun wrote:
On Sun, Dec 14, 2008 at 05:03:04PM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 04:24:43PM +0100, Diego Biurrun wrote:
On Mon, Dec 01, 2008 at 02:02:57PM +0100, Michael Niedermayer wrote:
maybe -i your_file -y -an -f null /dev/null
OK, I redid the benchmarks directly with ffmpeg on my K6-III, here are the results of 5 runs each, highest and lowest result discarded, all services stopped: [...] ffmpeg-split -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=284.858s bench: utime=284.110s bench: utime=283.610s
ffmpeg-vanilla -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=283.438s bench: utime=282.830s bench: utime=283.170s
The result seems to be no slowdown or a very minor slowdown below 1%.
So can I commit the next part of the split and continue working on it?
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
What is your suggestion?
I guess we would have to find out why its slower, once that is known we may be able to do something to avoid it. Finding this out though could be hard ... some things to try are listing the functions& sizes in the object file before and after the change there could be changes to what is inlined for example ... comparing profiler output, maybe this gives some hint spliting the patch down to see if the problematic part can be isolated [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Many things microsoft did are stupid, but not doing something just because microsoft did it is even more stupid. If everything ms did were stupid they would be bankrupt already.
On Sun, Dec 14, 2008 at 05:03:04PM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 04:24:43PM +0100, Diego Biurrun wrote:
On Mon, Dec 01, 2008 at 02:02:57PM +0100, Michael Niedermayer wrote:
On Mon, Dec 01, 2008 at 11:25:59AM +0100, Diego Biurrun wrote:
On Sat, Nov 29, 2008 at 05:28:27PM +0100, Michael Niedermayer wrote:
On Fri, Nov 28, 2008 at 09:46:19PM +0100, Diego Biurrun wrote:
On Fri, Nov 28, 2008 at 07:14:57PM +0100, Michael Niedermayer wrote: > On Fri, Nov 28, 2008 at 05:48:50PM +0100, Diego Biurrun wrote: > > Will this need to be benchmarked at some point? > > id say it cant hurt to benchmark it before commit
Hmmmmm
I ran the following benchmarks on my K6-III with two different samples:
http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg
I used the following MPlayer command lines to benchmark:
mplayer -quiet -frames 1000 -benchmark -nosound -vo xmga mplayer -quiet -frames 1000 -benchmark -nosound -vo null
I made five runs for each sample and vo and threw away the best and worst times. The results are attached.
Unfortunately there seems to be a slight slowdown with the cathedral sample, with Aladin it's a bit inconclusive.
Now I don't know how to continue from here. Were my benchmarking procedures sensible? (BTW, is there a good way to benchmark with ffmpeg directly?)
time ffmpeg ... >&/dev/null
Sure, but what options would I use?
maybe -i your_file -y -an -f null /dev/null
OK, I redid the benchmarks directly with ffmpeg on my K6-III, here are the results of 5 runs each, highest and lowest result discarded, all services stopped: [...] ffmpeg-split -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=284.858s bench: utime=284.110s bench: utime=283.610s
ffmpeg-vanilla -i cathedral-beta2-400extra-crop-avc.mp4 -an -f null -y -benchmark -v -1 /dev/null bench: utime=283.438s bench: utime=282.830s bench: utime=283.170s
The result seems to be no slowdown or a very minor slowdown below 1%.
So can I commit the next part of the split and continue working on it?
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
The slowdown is a whopping 0.37%, this is insane, differences between compiler versions or differences from completely random changes have a much bigger influence. This was a topic on IRC today: 17:10 <+uau> DonDiego: btw michael's latest complaint about the slowdown is completely baseless 17:11 <+uau> it's a 0.28 percent change 17:11 <@DonDiego> i was just going to run the numbers... 17:11 <+uau> if you make a few completely random changes to the code you can make it speed up much more :) 17:11 <@DonDiego> suggest some :) 17:17 <+uau> i tried adding av_noinline to one randomly picked function, that gave a consistent 0.6 percent speedup with a particular test file 17:17 <@DonDiego> which one? 17:19 <+ssvb> uau: ... also sometimes with a specific version of the compiler and specific cpu revision... 17:19 <+uau> then i tried adding a pointless global function that is not called from anywhere 17:19 <+uau> that made it 1.1% slower 17:20 <+uau> probably adding a bit different unused function would make it a similar amount faster 17:21 <+ssvb> there are different factors such as code alignment and cache aliasing which may introduce such random performance changes 17:22 <+ssvb> so if the code just shifts a bit, you may have speedup or slowdown in some completely unrelated part of program 17:22 <+uau> ssvb: that was my point, michael's complaint based on 0.28 percent performance change on one particular sample makes little sense 17:43 <+ssvb> uau: you can try to do the same benchmark with different versions of gcc and running it on different cpus, then consider average effect My opinion is that the benefits of the refactoring vastly outweigh concerns about less than 0.5% speed difference, especially if it is not clear that the speed difference is directly related to the changes. Diego
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
Why is this not ok, while disabling filter_mb_fast was OK? I measure a 25% performance increase on a sample here by re-enabling filter_mb_fast. Why is 0.5% here intolerable, but 25% there is just fine? Dark Shikari
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote:
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
Why is this not ok, while disabling filter_mb_fast was OK? I measure a 25% performance increase on a sample here by re-enabling filter_mb_fast. Why is 0.5% here intolerable, but 25% there is just fine?
One has to weigh the advantage against the disadvantage here, advanatge, of the change is largely cosmetic disadvantage is a 0.5% speed loss, after 10 such changes our decoder is 5% slower. And thouse claiming randomness, well, if its random a different gcc version shouldnt show the speed loss and in that case it could be reconsidered, though again if 4 out of 5 gcc versions show the speed loss it still doesnt look too good nor random. But if 2 out of 4 are slower and 2 faster it would be ok. About filter_mb_fast, this function did not work. And that was why it was disabled, its not a subtile change in the code that makes it slower but rather a clear disabling because its broken. When it is fixed the speed gain it provided will be back. also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast. doing optimzation well is not a matter of asking why its slower and then ignoring it when its not obvious, its a matter of picking the faster. Like in evolution, the question is not why some species is consistently 0.5% less effective than another just that it is. if one can find out why its 0.5% worse one may be able to find a solution that makes the wanted change and avoids the speedloss, but if not the change isnt ok. and about other random changes making the code 0.5% faster, these are VERY welcome, adding a noinline here, and inline there and such, really! [of course only when the patch is clean and doesnt mix unrelated stuff] [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB It is not what we do, but why we do it that matters.
On Mon, 2008-12-15 at 01:33 +0100, Michael Niedermayer wrote:
One has to weigh the advantage against the disadvantage
here, advanatge, of the change is largely cosmetic disadvantage is a 0.5% speed loss, after 10 such changes our decoder is 5% slower.
Real speed change yes, but 0.5% change in benchmark result tells you very little about whether speed changed or not ("real" speed could have become faster or slower).
And thouse claiming randomness, well, if its random a different gcc version shouldnt show the speed loss and in that case it could be reconsidered, though again if 4 out of 5 gcc versions show the speed loss it still doesnt look too good nor random. But if 2 out of 4 are slower and 2 faster it would be ok.
I haven't tested this particular change, but I have benchmarked various irrelevant changes to the h264 decoder earlier and they typically have effects at the percent or couple level (even when each resulting binary is benchmarked enough that the randomness caused by per run variation is much less).
also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
But benchmarking is not a reliable way to tell whether the h264 decoder really became 0.5% slower (at least you'd need a big collection of various possible "irrelevant" code changes and of compiler versions to benchmark against).
doing optimzation well is not a matter of asking why its slower and then ignoring it when its not obvious, its a matter of picking the faster. Like in evolution, the question is not why some species is consistently 0.5% less effective than another just that it is.
Evolution relies on a large collection of individuals with varying traits. It wouldn't work if it abandoned a trait when one individual with it dies.
if one can find out why its 0.5% worse one may be able to find a solution that makes the wanted change and avoids the speedloss, but if not the change isnt ok.
What about all the other changes you applied that made it 0.5% slower? Since the speed varies back and forth with most changes, about half of them (other than clearly beneficial optimizations) have made it slower, and a significant portion of those by 0.5% or more. Even changes in other decoders can affect h264 speed that much (probably some memory aliasing effect).
and about other random changes making the code 0.5% faster, these are VERY welcome, adding a noinline here, and inline there and such, really! [of course only when the patch is clean and doesnt mix unrelated stuff]
Such random changes have about 50% chance of being beneficial on any other combination of compiler, machine and exact FFmpeg source used. The changes I tested probably wouldn't have the same effect in ffplay as in my test under MPlayer. The results really are essentially "random".
On Mon, Dec 15, 2008 at 04:52:34AM +0200, Uoti Urpala wrote:
On Mon, 2008-12-15 at 01:33 +0100, Michael Niedermayer wrote:
One has to weigh the advantage against the disadvantage
here, advanatge, of the change is largely cosmetic disadvantage is a 0.5% speed loss, after 10 such changes our decoder is 5% slower.
Real speed change yes, but 0.5% change in benchmark result tells you very little about whether speed changed or not ("real" speed could have become faster or slower).
Please see my mail i just posted for an actual test of what can reliably be detected. [...]
also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
But benchmarking is not a reliable way to tell whether the h264 decoder really became 0.5% slower (at least you'd need a big collection of various possible "irrelevant" code changes and of compiler versions to benchmark against).
Please see my mail i just posted for an actual test of what can reliably be detected.
doing optimzation well is not a matter of asking why its slower and then ignoring it when its not obvious, its a matter of picking the faster. Like in evolution, the question is not why some species is consistently 0.5% less effective than another just that it is.
Evolution relies on a large collection of individuals with varying traits. It wouldn't work if it abandoned a trait when one individual with it dies.
are we speaking about the same thing? it seems not
if one can find out why its 0.5% worse one may be able to find a solution that makes the wanted change and avoids the speedloss, but if not the change isnt ok.
What about all the other changes you applied that made it 0.5% slower?
which change?
Since the speed varies back and forth with most changes, about half of them (other than clearly beneficial optimizations) have made it slower, and a significant portion of those by 0.5% or more. Even changes in other decoders can affect h264 speed that much (probably some memory aliasing effect).
The speed very likely does vary yes, if its half faster half slower is something you are guessing not knowing i suspect. Same for the relation to 0.5%
and about other random changes making the code 0.5% faster, these are VERY welcome, adding a noinline here, and inline there and such, really! [of course only when the patch is clean and doesnt mix unrelated stuff]
Such random changes have about 50% chance of being beneficial on any other combination of compiler, machine and exact FFmpeg source used.
The changes I tested probably wouldn't have the same effect in ffplay as in my test under MPlayer.
The results really are essentially "random".
So what? is it "probably" or "really are essentially" Diego tested the change in ffmpeg and mplayer, the results where the same you just said you didnt test except one case (mplayer). Now you extrapolate from 1 case that it would be random. Are you studying cosmology by chance? [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Observe your enemies, for they first find out your faults. -- Antisthenes
On Tue, 2008-12-16 at 00:43 +0100, Michael Niedermayer wrote:
On Mon, Dec 15, 2008 at 04:52:34AM +0200, Uoti Urpala wrote:
On Mon, 2008-12-15 at 01:33 +0100, Michael Niedermayer wrote:
One has to weigh the advantage against the disadvantage
here, advanatge, of the change is largely cosmetic disadvantage is a 0.5% speed loss, after 10 such changes our decoder is 5% slower.
Real speed change yes, but 0.5% change in benchmark result tells you very little about whether speed changed or not ("real" speed could have become faster or slower).
Please see my mail i just posted for an actual test of what can reliably be detected.
Your other mail seems to be about something else, namely benchmarking a particular binary. You can benchmark the performance of one binary to a precision higher than 0.5%; but the performance of that binary will have a component determined by "random" compile-time effects that is larger than 0.5%. As I wrote in the part of my earlier mail you didn't quote: "I have benchmarked various irrelevant changes to the h264 decoder earlier and they typically have effects at the percent or couple level (even when each resulting binary is benchmarked enough that the randomness caused by per run variation is much less)". You're only measuring the "randomness caused by per run variation", which I already said can be much less. The "random" variation I'm talking about is NOT between different runs of the same binary, but between slightly different compiles of the same code. For example using a slightly different compiler version, changing code whose performance doesn't matter or which is completely unused, or even changing code in another translation unit from which nothing is executed in your benchmark. Such "irrelevant" changes can change performance by a couple of percent, and any real change can change the "random" effects in addition to its consistent performance effects. You can benchmark change X to the h264 decoder and note that it makes h264 decoding 1% slower. Then someone changes the vorbis decoder, and after that change X makes h264 decoding 1% faster on your computer. On someone else's machine the effects could be reverse (first X looks beneficial, then harmful).
[...]
also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
But benchmarking is not a reliable way to tell whether the h264 decoder really became 0.5% slower (at least you'd need a big collection of various possible "irrelevant" code changes and of compiler versions to benchmark against).
Please see my mail i just posted for an actual test of what can reliably be detected.
It shows that the random result from the compile can be detected. That doesn't mean it could detect the real quality of the code before the compile randomness is added to it. Also real code would likely have more runtime randomness too than the code in your benchmark (at least memory patterns would vary more depending on heap and stack location randomization in a real program).
doing optimzation well is not a matter of asking why its slower and then ignoring it when its not obvious, its a matter of picking the faster. Like in evolution, the question is not why some species is consistently 0.5% less effective than another just that it is.
Evolution relies on a large collection of individuals with varying traits. It wouldn't work if it abandoned a trait when one individual with it dies.
are we speaking about the same thing? it seems not
If you kept a huge collection of programs with various features (one with patches X, Y and Z applied, another with X, Z and W applied, and a few thousand other variations) and occasionally pruned the overall slowest variations then that could work like evolution. Abandoning a patch because one binary with it applied is slower would be more like evolution abandoning a gene because one individual with it died, which wouldn't work.
if one can find out why its 0.5% worse one may be able to find a solution that makes the wanted change and avoids the speedloss, but if not the change isnt ok.
What about all the other changes you applied that made it 0.5% slower?
which change?
Lots of them as I explained below (exactly which ones they are depend on whose machine they're benchmarked on...).
Since the speed varies back and forth with most changes, about half of them (other than clearly beneficial optimizations) have made it slower, and a significant portion of those by 0.5% or more. Even changes in other decoders can affect h264 speed that much (probably some memory aliasing effect).
The speed very likely does vary yes, if its half faster half slower is something you are guessing not knowing i suspect. Same for the relation to 0.5%
Half faster half slower is logical effect for "random" changes. The relation to 0.5% is based on what I've seen in practice as I've benchmarked h264 decoding in different versions.
and about other random changes making the code 0.5% faster, these are VERY welcome, adding a noinline here, and inline there and such, really! [of course only when the patch is clean and doesnt mix unrelated stuff]
Such random changes have about 50% chance of being beneficial on any other combination of compiler, machine and exact FFmpeg source used.
The changes I tested probably wouldn't have the same effect in ffplay as in my test under MPlayer.
The results really are essentially "random".
So what? is it "probably" or "really are essentially"
You're confusing the subjects of the sentences. There are no two statements about a single "it" but two separate statements. The performance effect of that particular change probably wouldn't be the same under ffplay (but there's a chance it would be; it depends on which way the "random" roll would land). And the results are essentially "random" (though strictly speaking deterministic, but "random" in the same sense as the output of a hash function).
Diego tested the change in ffmpeg and mplayer, the results where the same you just said you didnt test except one case (mplayer). Now you extrapolate from 1 case that it would be random. Are you studying cosmology by chance?
I do not extrapolate from one case. I determine it based on several prior measurements of the behavior of the h264 decoder performance; and it doesn't require much extrapolating to say that the performance variation of the current decoder probably isn't much smaller than that measured before.
On Mon, Dec 15, 2008 at 01:33:49AM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote:
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
Why is this not ok, while disabling filter_mb_fast was OK? I measure a 25% performance increase on a sample here by re-enabling filter_mb_fast. Why is 0.5% here intolerable, but 25% there is just fine?
One has to weigh the advantage against the disadvantage
here, advanatge, of the change is largely cosmetic
You consider code maintainability cosmetic?
disadvantage is a 0.5% speed loss, after 10 such changes our decoder is 5% slower.
The separation of the two decoders can only be done once and will only be done once. There is absolutely no danger for this to become a slippery slope.
And thouse claiming randomness, well, if its random a different gcc version shouldnt show the speed loss and in that case it could be reconsidered, though again if 4 out of 5 gcc versions show the speed loss it still doesnt look too good nor random. But if 2 out of 4 are slower and 2 faster it would be ok.
Do you claim that H.264 decoding performance is not sensitive to random changes and even to changes unrelated to H.264?
also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
Well, our H.264 decoder is not the fastest around and it still has areas where it can be improved vastly. So skipping refactoring for 0.3% speed in one sample - don't forget that the other one improved - is premature optimization. But most importantly I think this obsession with speed at all costs is hurting people's motivation and losing us much more than 0.3% performance. I know that I already spent far too much time on this split without real progress and my motivation is going in one direction only: down. There have been tons of changes to h264.c that were not subjected to the speed inquisition, everything PAFF-related jumps to my mind for example. The H.264/SVQ3 split is being singled out for unfathomable reasons. Probably because I made the error of waking sleeping dogs by running benchmarks in the first place. Below is a slightly edited IRC log from yesterday night. It should speak volumes about developer motivation and places where *substantial* speed improvements can be achieved. Diego 00:18 <@Dark_Shikari> DonDiego: while you're at it, split h264.c into meaningful sections so that you don't have all of h264 decoding in one gigantic file 00:18 <@Dark_Shikari> residual decoding should go in cabac and cavlc c files, respectively 00:18 <@DonDiego> Dark_Shikari: why don't you split the file? 00:19 <@DonDiego> you know h.264 much better than i do.. 00:19 <@Dark_Shikari> DonDiego: I don't maintain it, and I'm not prepared to go through the nightmare of trying to convince michael 00:19 <@DonDiego> what nightmare? 00:20 <@Dark_Shikari> you just posted above about your annoyance at michael not accepting the SVQ3 split 00:20 <@Dark_Shikari> which is far far far simpler than what I'm describing 00:20 <@Dark_Shikari> you know what I'm talking about 00:20 <@DonDiego> i wanted it spelled out 00:20 <@DonDiego> i don't think michael is opposed to the split you suggest 00:20 <@DonDiego> he knows that h264.c is far too large for its own good 00:21 <@Dark_Shikari> sure, but he'll probably find the split makes things like 0.1% slower 00:21 <@Dark_Shikari> and reject it 00:21 <@DonDiego> just his obsession with performance is insane 00:21 <@DonDiego> well, let me duke that out with him, if i manage to convince him, you can go one step further 00:22 <@Dark_Shikari> well the odd thing about it is that for example x264 is far far far more obsessed with performance than michael's ffh264 00:23 <@Dark_Shikari> ... yet we don't have to massively sacrifice code maintainability to do it 00:23 <@Dark_Shikari> so I'm not exactly sure why he's like this 00:23 <@Dark_Shikari> if he really cared that much he'd focus on actually making it faster.. 00:23 <@superdump> more/better simd you mean? 00:24 <@Dark_Shikari> better simd, less overhead 00:24 <@Dark_Shikari> simd-wise, we've already done it for him 00:24 <@superdump> make the point 00:25 <@Dark_Shikari> ok, then I will 00:25 <@superdump> but you don't have the time/desire to port it over to ffh264? 00:25 <@Dark_Shikari> I don't have the time/desire to deal with michael when doing it 00:25 <@Dark_Shikari> the deblock code doesn't even NEED porting 00:26 <@Dark_Shikari> it can probably be pasted right in with zero changes 00:26 <@Dark_Shikari> there are just two problems with things: 00:26 <@Dark_Shikari> 1) Its GPL, this is not changing 00:26 <@Dark_Shikari> 2) It WILL CRASH with an unaligned stack, period 00:26 <@Dark_Shikari> so nobody will be able to get away with miscompiling lavc anymore 00:28 <@superdump> Dark_Shikari: do you have any kind of good estimate as to how much performance should be gained by reusing the x264 asm code? 00:29 <@Dark_Shikari> superdump: let me do the math 00:29 <@Dark_Shikari> all numbers will be using nehalem, as that's what Ihave to bench on with linux 64-bit 00:30 <@Dark_Shikari> actually, maybe not, as 64-bit doesn't compile x264's mmx asm for deblock 00:30 <@Dark_Shikari> as no 64-bit machine uses it 00:34 <@Dark_Shikari> ok, let us assume that x264's mmx deblock is 10% faster than lavc's inherently 00:34 <@Dark_Shikari> numbers on mmx vs sse in x264: 00:34 <@Dark_Shikari> sse2 luma V deblock :2.3x faster than mmx 00:35 <@superdump> on nehalem which is crazy fast 00:35 <@Dark_Shikari> SSE2 luma H deblock: 1.34x faster 00:35 <@Dark_Shikari> SSE2 luma H intra deblock: 1.44x faster 00:35 <@Dark_Shikari> SSE2 luma V intra deblock: 2.19x faster 00:35 <@Dark_Shikari> now let's grab an ffh264 profile 00:38 <@Dark_Shikari> 32-bit test, as the asm code has no mmx equivalent on 64-bit 00:38 <@Dark_Shikari> as the 64-bit asm uses, like, all of the registers 00:43 <@Dark_Shikari> oh btw, what's a good way to benchmark just decoding 00:43 <@Dark_Shikari> -f null? 00:43 <@mru> yes, and -an too 00:44 <@mru> if the clip has audio 00:44 <@Dark_Shikari> decoding 30 seconds of 1080p 00:45 <@Dark_Shikari> what the hell, filter_mb is 15.1% of runtime? 00:47 <@Dark_Shikari> 2485 15.1165 ffmpeg_g ffmpeg_g filter_mb 00:47 <@Dark_Shikari> 1576 9.5870 ffmpeg_g ffmpeg_g hl_decode_mb_simple 00:47 <@Dark_Shikari> 1446 8.7962 ffmpeg_g ffmpeg_g fill_caches 00:47 <@Dark_Shikari> 797 4.8482 ffmpeg_g ffmpeg_g put_h264_chroma_mc8_ssse3_rnd 00:48 <@Dark_Shikari> how can that be right? shouldn't filter_mb_fast be being run? 00:49 <@Dark_Shikari> Oh. I see where the problem comes from. 00:49 <@Dark_Shikari> h->pps.chroma_qp_diff 00:49 <@Dark_Shikari> filter_mb is run instead of filter_mb_fast when chroma_qp_offset is set. 00:49 <+uau> how did you profile it? 00:49 <@mru> evil sample? 00:49 <@Dark_Shikari> oprofile 00:49 <@Dark_Shikari> no no 00:49 <@Dark_Shikari> mru: x264 uses chroma qp offset by default 00:49 <+uau> how did you compile it? 00:49 <@Dark_Shikari> for over 3 months now 00:49 <@Dark_Shikari> uau: normally 00:49 <@Yuvi> I thought michael disabled filter_mb_fast a while ago 00:49 <+uau> there probably is no filter_mb_fast with default options 00:50 <@Dark_Shikari> no, uau, there is 00:50 <@Dark_Shikari> .... the issue is, lavc has an extremely nice loop filter calculation SIMD function 00:50 <@Dark_Shikari> written in MMX 00:50 <@Dark_Shikari> it is awesome 00:50 <@Dark_Shikari> ... but it isn't being used 00:50 <@Dark_Shikari> because it doesn't run if there's a chroma qp offset 00:50 <+uau> why would there be such a function? 00:50 <+uau> it's static and used in only one place 00:50 <@Dark_Shikari> this is retarded, since its trivial to deal with a chroma qp offset 00:50 <@Dark_Shikari> uau: huh? because its much faster 00:50 <+uau> i mean the binary 00:51 <+uau> why would the binary have such a function 00:51 <@Dark_Shikari> what do you mean 00:51 <@Dark_Shikari> what binary 00:51 <@Dark_Shikari> it isn't inlined, no 00:51 <+uau> why not? 00:51 <@Dark_Shikari> gcc does not inline everything that's only used once and is static 00:52 <+uau> it is inlined here 00:53 <+uau> no filter_mb_fast in h264.o 00:53 <@Dark_Shikari> ok, you're right, it isn't here either--but it isn't being used regardless 00:53 <@Dark_Shikari> I'll send an email to the ML about it 00:54 <@Dark_Shikari> actually, wait, this can't be right 00:54 <@Dark_Shikari> chroma_qp_diff is when the chroma qp offsets differ 00:54 <@Dark_Shikari> NOT when there is an offset 00:54 <@Dark_Shikari> oh, I see. michael did disable filter_mb_fast, singlehandedly crippling decode speed. 00:54 <@Dark_Shikari> if(mb_x==0 || mb_y==mb_y_firstrow || !s->dsp.h264_loop_filter_strength || h->pps.chroma_qp_diff || 00:54 <@Dark_Shikari> 1 || 00:54 <@Dark_Shikari> gg michael 00:55 <@Dark_Shikari> way to write one of the best asm functions I've ever seen and then NOT USE IT 00:55 <@Dark_Shikari> that's why it isn't in your binary btw, it's never called 00:55 <@Dark_Shikari> or, more exactly, it instantly terminates 00:56 <@Dark_Shikari> fps went from 44 to 55 when re-enabling filter_mb_fast. 00:57 <@superdump> why is it disabled? 01:00 <@mru> wasn't there a hard to find bug or something? 01:01 <@Dark_Shikari> dunno, but I would prefer slightly wrong but fast decoding 01:01 <@Dark_Shikari> bitexact flags exist for a reason 01:03 <@Dark_Shikari> also, I find it unreasonable that fill_caches takes almost 10% of decoding time
Hi! Diego Biurrun <diego <at> biurrun.de> writes:
Well, our H.264 decoder is not the fastest around
CoreAVC was (much) slower on single core CPUs when I last tested. I've never seen different results. Or are you referring to another decoder?
and it still has areas where it can be improved vastly. So skipping refactoring for 0.3% speed in one sample - don't forget that the other one improved - is premature optimization.
[...]
There have been tons of changes to h264.c that were not subjected to the speed inquisition, everything PAFF-related jumps to my mind for example.
Everything PAFF-related was absolutely necessary to make the decoder (more) usable. There was a huge amount of bugreports about it. Carl Eugen
On Mon, Dec 15, 2008 at 4:53 PM, Carl Eugen Hoyos <cehoyos at ag.or.at> wrote:
Hi!
Diego Biurrun <diego <at> biurrun.de> writes:
Well, our H.264 decoder is not the fastest around
CoreAVC was (much) slower on single core CPUs when I last tested. I've never seen different results. Or are you referring to another decoder?
Last I remember testing, it was faster on almost all CPUs except the most recent Intel ones (where SSE is so fast that CoreAVC's lack of SSE becomes rather painful). The interesting thing is that CoreAVC is actually surprisingly similar to ffh264 in terms of performance features--I am still not entirely sure why it is at all faster. It may simply be less overhead due to better organization, or perhaps its prefetching and better cache handling. Dark Shikari
On Tue, Dec 16, 2008 at 01:34:00AM +0100, Diego Biurrun wrote:
On Mon, Dec 15, 2008 at 01:33:49AM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote:
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
Why is this not ok, while disabling filter_mb_fast was OK? I measure a 25% performance increase on a sample here by re-enabling filter_mb_fast. Why is 0.5% here intolerable, but 25% there is just fine?
One has to weigh the advantage against the disadvantage
here, advanatge, of the change is largely cosmetic
You consider code maintainability cosmetic?
no, but i consider the svq3/h264 split largely cosmetic, these 2 are in seperate files already. I do want the split, but iam not convinced that we have to loose 0.5% to have it.
disadvantage is a 0.5% speed loss, after 10 such changes our decoder is 5% slower.
The separation of the two decoders can only be done once and will only be done once. There is absolutely no danger for this to become a slippery slope.
Each individual change can be done just once.
And thouse claiming randomness, well, if its random a different gcc version shouldnt show the speed loss and in that case it could be reconsidered, though again if 4 out of 5 gcc versions show the speed loss it still doesnt look too good nor random. But if 2 out of 4 are slower and 2 faster it would be ok.
Do you claim that H.264 decoding performance is not sensitive to random changes and even to changes unrelated to H.264?
no i did not claim that
also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
Well, our H.264 decoder is not the fastest around and it still has areas where it can be improved vastly. So skipping refactoring for 0.3% speed in one sample - don't forget that the other one improved - is premature optimization.
But most importantly I think this obsession with speed at all costs is hurting people's motivation and losing us much more than 0.3% performance. I know that I already spent far too much time on this split without real progress and my motivation is going in one direction only: down.
no problem, ill spend the 30min to find out why its slower and see if it can be avoided. if we can get 0.5% speed by 30mins our decoder would be 24% faster per day premature or not ...
There have been tons of changes to h264.c that were not subjected to the speed inquisition, everything PAFF-related jumps to my mind for example.
benchmarking past changes is welcome, i surely think we should do that one day and look into speedlosses.
The H.264/SVQ3 split is being singled out for unfathomable reasons. Probably because I made the error of waking sleeping dogs by running benchmarks in the first place.
Below is a slightly edited IRC log from yesterday night. It should speak volumes about developer motivation and places where *substantial* speed improvements can be achieved.
I know dark shikari is not porting anything and i know he blames random things for why he is not doing anything. I cant help but if he is not porting code then this is his decission and he really should not blame anyone but himself. Just look at the mailing list, what is it that we have A. many patches from dark shikari that i rejected? or. B. few patches from dark shikari that have comments and questions in replies to which he never even bothered to reply? Its not that he is disagreeing about some comment from me. If there is some rule, or otherwise he dislikes, he has not said this on the ML yet. I cant read his mind on what the problem is if there is any. Also i do not think that he would be ok with others commiting poor code randomly to x264 ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB While the State exists there can be no freedom; when there is freedom there will be no State. -- Vladimir Lenin
On Tue, Dec 16, 2008 at 02:26:07AM +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 01:34:00AM +0100, Diego Biurrun wrote:
On Mon, Dec 15, 2008 at 01:33:49AM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote:
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok [...] also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
Well, our H.264 decoder is not the fastest around and it still has areas where it can be improved vastly. So skipping refactoring for 0.3% speed in one sample - don't forget that the other one improved - is premature optimization.
But most importantly I think this obsession with speed at all costs is hurting people's motivation and losing us much more than 0.3% performance. I know that I already spent far too much time on this split without real progress and my motivation is going in one direction only: down.
no problem, ill spend the 30min to find out why its slower and see if it can be avoided. if we can get 0.5% speed by 30mins our decoder would be 24% faster per day premature or not ...
i just realized you didnt post the patch, please do i cant look into it without having it :) also a link to the slower becomig file would be usefull [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Asymptotically faster algorithms should always be preferred if you have asymptotical amounts of data
On Tue, Dec 16, 2008 at 02:36:53AM +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 02:26:07AM +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 01:34:00AM +0100, Diego Biurrun wrote:
On Mon, Dec 15, 2008 at 01:33:49AM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote:
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok [...] also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
Well, our H.264 decoder is not the fastest around and it still has areas where it can be improved vastly. So skipping refactoring for 0.3% speed in one sample - don't forget that the other one improved - is premature optimization.
But most importantly I think this obsession with speed at all costs is hurting people's motivation and losing us much more than 0.3% performance. I know that I already spent far too much time on this split without real progress and my motivation is going in one direction only: down.
no problem, ill spend the 30min to find out why its slower and see if it can be avoided. if we can get 0.5% speed by 30mins our decoder would be 24% faster per day premature or not ...
i just realized you didnt post the patch, please do i cant look into it without having it :) also a link to the slower becomig file would be usefull
I did post a link to the patch. I did not attach it because it is 118kB: http://www1.mplayerhq.hu/~diego/split.diff Here is the sample that "slows down": http://samples.mplayerhq.hu/V-codecs/h264/cathedral-beta2-400extra-crop-avc.... This one gets faster: http://samples.mplayerhq.hu/V-codecs/h264/Aladin.mpg Probably it would be interesting to test some other samples... Diego
On Tue, 2008-12-16 at 09:43 +0100, Diego Biurrun wrote:
I did post a link to the patch. I did not attach it because it is 118kB:
I see about 8% slowdown. That is larger than the usual random performance changes.
Uoti Urpala wrote:
On Tue, 2008-12-16 at 09:43 +0100, Diego Biurrun wrote:
I did post a link to the patch. I did not attach it because it is 118kB:
I see about 8% slowdown. That is larger than the usual random performance changes.
Using which CPU, compiler, and sample? -- M?ns Rullg?rd mans at mansr.com
On Tue, 2008-12-16 at 10:58 +0000, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 09:43 +0100, Diego Biurrun wrote:
I did post a link to the patch. I did not attach it because it is 118kB:
I see about 8% slowdown. That is larger than the usual random performance changes.
Using which CPU, compiler, and sample?
Q9550. gcc 4.3.2 showed 8%, current development gcc somewhat less. The results were similar on 3 different samples, one of which was listed in Diego's mail.
Uoti Urpala <uoti.urpala at pp1.inet.fi> writes:
On Tue, 2008-12-16 at 10:58 +0000, M?ns Rullg?rd wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 09:43 +0100, Diego Biurrun wrote:
I did post a link to the patch. I did not attach it because it is 118kB:
I see about 8% slowdown. That is larger than the usual random performance changes.
Using which CPU, compiler, and sample?
Q9550. gcc 4.3.2 showed 8%, current development gcc somewhat less. The results were similar on 3 different samples, one of which was listed in Diego's mail.
I'm getting about 4.4% slowdown on Q6700. Here's a comparison of oprofile reports for the two builds, sorted by absolute change in relative time per symbol: +1.7296 7.8030 9.5326 fill_caches -0.7182 16.3912 15.6730 filter_mb -0.3785 6.9084 6.5299 hl_decode_mb_simple +0.3512 3.9060 4.2572 decode_mb_skip -0.2795 2.4954 2.2159 pred_direct_motion -0.2478 2.6410 2.3932 get_cabac_noinline -0.2401 3.9328 3.6927 put_h264_qpel8or16_v_lowpass_sse2 -0.2124 6.3883 6.1759 decode_mb_cabac -0.1782 3.4897 3.3115 decode_cabac_residual_nondc -0.1636 1.1769 1.0133 filter_mb_edgev -0.1536 4.8790 4.7254 put_h264_chroma_mc8_ssse3_rnd -0.1452 1.6699 1.5247 decode_cabac_mb_mvd -0.1450 1.8797 1.7347 h264_h_loop_filter_chroma_mmx2 -0.1208 2.7630 2.6422 put_h264_qpel16_h_lowpass_l2_ssse3 -0.1059 2.7304 2.6245 h264_h_loop_filter_luma_mmx2 -0.1029 4.0421 3.9392 put_h264_qpel8or16_hv_lowpass_ssse3 -0.0858 2.0388 1.9530 prefetch_mmx2 -0.0853 2.1108 2.0255 clear_blocks_mmx -0.0805 1.4704 1.3899 put_h264_qpel16_mc00_sse2 +0.0729 0.5442 0.6171 decode_cabac_mb_skip +0.0689 1.1119 1.1808 filter_mb_edgeh +0.0539 0.3742 0.4281 filter_mb_fast -0.0516 1.3796 1.3280 put_h264_qpel8_h_lowpass_ssse3 -0.0504 2.0254 1.9750 mc_dir_part -- M?ns Rullg?rd mans at mansr.com
2008/12/16 Uoti Urpala <uoti.urpala at pp1.inet.fi>:
On Tue, 2008-12-16 at 09:43 +0100, Diego Biurrun wrote:
I did post a link to the patch. I did not attach it because it is 118kB:
I see about 8% slowdown. That is larger than the usual random performance changes.
CPU: Core Duo 1.83GHz 2MB L2 OS: Mac OS 10.5.6 Command line: time ffmpeg -i ~/samples/h264/BigBuckBunny.mkv -t 30 -an -f null /dev/null >& /dev/null Vanilla: real 0m41.431s user 0m40.595s sys 0m0.438s real 0m41.260s user 0m40.582s sys 0m0.408s real 0m41.191s user 0m40.549s sys 0m0.397s Split: real 0m43.197s user 0m42.405s sys 0m0.426s real 0m43.170s user 0m42.406s sys 0m0.438s real 0m43.069s user 0m42.370s sys 0m0.403s Rob
2008/12/16 Robert Swain <robert.swain at gmail.com>:
2008/12/16 Uoti Urpala <uoti.urpala at pp1.inet.fi>:
On Tue, 2008-12-16 at 09:43 +0100, Diego Biurrun wrote:
I did post a link to the patch. I did not attach it because it is 118kB:
I see about 8% slowdown. That is larger than the usual random performance changes.
CPU: Core Duo 1.83GHz 2MB L2 OS: Mac OS 10.5.6 Command line: time ffmpeg -i ~/samples/h264/BigBuckBunny.mkv -t 30 -an -f null /dev/null >& /dev/null
Vanilla:
real 0m41.431s user 0m40.595s sys 0m0.438s
real 0m41.260s user 0m40.582s sys 0m0.408s
real 0m41.191s user 0m40.549s sys 0m0.397s
Split:
real 0m43.197s user 0m42.405s sys 0m0.426s
real 0m43.170s user 0m42.406s sys 0m0.438s
real 0m43.069s user 0m42.370s sys 0m0.403s
With i686-apple-darwin9-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5566). Rob
On Tue, Dec 16, 2008 at 09:43:35AM +0100, Diego Biurrun wrote: [...]
I did post a link to the patch. I did not attach it because it is 118kB:
--2010-01-10 23:24:57-- http://www1.mplayerhq.hu/~diego/split.diff Resolving www1.mplayerhq.hu... 213.144.138.186 Connecting to www1.mplayerhq.hu|213.144.138.186|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2010-01-10 23:24:57 ERROR 404: Not Found. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The greatest way to live with honor in this world is to be what we pretend to be. -- Socrates
On Sun, Jan 10, 2010 at 11:28:30PM +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 09:43:35AM +0100, Diego Biurrun wrote: [...]
I did post a link to the patch. I did not attach it because it is 118kB:
--2010-01-10 23:24:57-- http://www1.mplayerhq.hu/~diego/split.diff Resolving www1.mplayerhq.hu... 213.144.138.186 Connecting to www1.mplayerhq.hu|213.144.138.186|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2010-01-10 23:24:57 ERROR 404: Not Found.
I don't have the patch in a place where I can find it right now. It basically changes the .h file into a .c file IIRC. Just read the mails, there were some sideeffects that affected the gcc random code generator. Diego
On Sun, Jan 10, 2010 at 11:39:48PM +0100, Diego Biurrun wrote:
On Sun, Jan 10, 2010 at 11:28:30PM +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 09:43:35AM +0100, Diego Biurrun wrote: [...]
I did post a link to the patch. I did not attach it because it is 118kB:
--2010-01-10 23:24:57-- http://www1.mplayerhq.hu/~diego/split.diff Resolving www1.mplayerhq.hu... 213.144.138.186 Connecting to www1.mplayerhq.hu|213.144.138.186|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2010-01-10 23:24:57 ERROR 404: Not Found.
I don't have the patch in a place where I can find it right now. It basically changes the .h file into a .c file IIRC.
ok, ill try to redo it then
Just read the mails, there were some sideeffects that affected the gcc random code generator.
thats the most likely explanation but not the only possible one [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB When the tyrant has disposed of foreign enemies by conquest or treaty, and there is nothing more to fear from them, then he is always stirring up some war or other, in order that the people may require a leader. -- Plato
On Tue, 16 Dec 2008 02:26:07 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 01:34:00AM +0100, Diego Biurrun wrote:
On Mon, Dec 15, 2008 at 01:33:49AM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote: also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
Well, our H.264 decoder is not the fastest around and it still has areas where it can be improved vastly. So skipping refactoring for 0.3% speed in one sample - don't forget that the other one improved - is premature optimization.
But most importantly I think this obsession with speed at all costs is hurting people's motivation and losing us much more than 0.3% performance. I know that I already spent far too much time on this split without real progress and my motivation is going in one direction only: down.
no problem, ill spend the 30min to find out why its slower and see if it can be avoided. if we can get 0.5% speed by 30mins our decoder would be 24% faster per day premature or not ...
this is the reason that ffmpeg is the fastest decoder for a lot of formats. the developers take great care to benchmark each change and make sure they didnt lose any speed. diego : you can speed up mplayer's h264 decoding in another way by getting the coreavc-for-linux project merged. -compn
On Mon, Dec 15, 2008 at 08:49:37PM -0500, compn wrote:
On Tue, 16 Dec 2008 02:26:07 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 01:34:00AM +0100, Diego Biurrun wrote:
On Mon, Dec 15, 2008 at 01:33:49AM +0100, Michael Niedermayer wrote:
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote: also ive developed and optimized other codecs like the mpeg4 variants based on this concept and they are pretty much the fastest around. Had i applied random changes that made the code by 0.5% slower they would not be as fast.
Well, our H.264 decoder is not the fastest around and it still has areas where it can be improved vastly. So skipping refactoring for 0.3% speed in one sample - don't forget that the other one improved - is premature optimization.
But most importantly I think this obsession with speed at all costs is hurting people's motivation and losing us much more than 0.3% performance. I know that I already spent far too much time on this split without real progress and my motivation is going in one direction only: down.
no problem, ill spend the 30min to find out why its slower and see if it can be avoided. if we can get 0.5% speed by 30mins our decoder would be 24% faster per day premature or not ...
this is the reason that ffmpeg is the fastest decoder for a lot of formats. the developers take great care to benchmark each change and make sure they didnt lose any speed.
No, it's simply not true that each change is benchmarked.
diego : you can speed up mplayer's h264 decoding in another way by getting the coreavc-for-linux project merged.
I much prefer free software and a fast ffh264. Diego
On Tue, 2008-12-16 at 02:26 +0100, Michael Niedermayer wrote: [...]
There have been tons of changes to h264.c that were not subjected to the speed inquisition, everything PAFF-related jumps to my mind for example.
benchmarking past changes is welcome, i surely think we should do that one day and look into speedlosses.
Now this isn't not exactly fair, is it? Those benchmarks should have occurred before the patches got applied. You can't expect people to accept there patches being rejected for some speed loss, while others got committed without benchmarking. Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines). With friendly regards, Takis
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
On Tue, 2008-12-16 at 02:26 +0100, Michael Niedermayer wrote: [...]
There have been tons of changes to h264.c that were not subjected to the speed inquisition, everything PAFF-related jumps to my mind for example.
benchmarking past changes is welcome, i surely think we should do that one day and look into speedlosses.
Now this isn't not exactly fair, is it? Those benchmarks should have occurred before the patches got applied. You can't expect people to accept there patches being rejected for some speed loss, while others got committed without benchmarking.
*if you care why did you not complain when whatever changes you speak of got commited? And if you dont care, why do you complain now? *Code has to work before one should optimize it, i think one hardly can claim our h264 decoder was working when it decoded just a small fraction of h264 streams correctly
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines).
What evidence is there to support this claim? if i remove some unneeded code and that results in a 0.5% gain on one machine chances are it also does on most others, its not as if the removial of code will likely make it slower. Besides, instead of telling me that i should accept some patch that has not fully understood sideeffects. Dont you think it would be more productive if you did work on some code, be that this patch or another? [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I do not agree with what you have to say, but I'll defend to the death your right to say it. -- Voltaire
On Tue, Dec 16, 2008 at 01:31:32PM +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
On Tue, 2008-12-16 at 02:26 +0100, Michael Niedermayer wrote: [...]
There have been tons of changes to h264.c that were not subjected to the speed inquisition, everything PAFF-related jumps to my mind for example.
benchmarking past changes is welcome, i surely think we should do that one day and look into speedlosses.
Now this isn't not exactly fair, is it? Those benchmarks should have occurred before the patches got applied. You can't expect people to accept there patches being rejected for some speed loss, while others got committed without benchmarking.
*if you care why did you not complain when whatever changes you speak of got commited? And if you dont care, why do you complain now?
He is complaining about the relative fairness of the treatment. Other changes are not subjected to extensive benchmarking requirements, this one is.
Besides, instead of telling me that i should accept some patch that has not fully understood sideeffects. Dont you think it would be more productive if you did work on some code, be that this patch or another?
That's a bad argument, it would also be more productive if you fixed filter_mb_fast instead of arguing here or ported optimizations from x264 or ... Diego
Diego Biurrun wrote:
On Tue, Dec 16, 2008 at 01:31:32PM +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
On Tue, 2008-12-16 at 02:26 +0100, Michael Niedermayer wrote: [...]
There have been tons of changes to h264.c that were not subjected to the speed inquisition, everything PAFF-related jumps to my mind for example.
benchmarking past changes is welcome, i surely think we should do that one day and look into speedlosses.
Now this isn't not exactly fair, is it? Those benchmarks should have occurred before the patches got applied. You can't expect people to accept there patches being rejected for some speed loss, while others got committed without benchmarking.
*if you care why did you not complain when whatever changes you speak of got commited? And if you dont care, why do you complain now?
He is complaining about the relative fairness of the treatment. Other changes are not subjected to extensive benchmarking requirements, this one is.
This is not a question of fairness. If patches have been accepted in the past without benchmarking (pure bugfixes don't count for obvious reasons), that is unfortunate. Such oversights in the past must not be used to justify continued sloppiness. Quality comes as a result of strictness, not from fairness. While I do consider Diego's cleanup efforts worthwhile, this particular patch undoubtedly still has some questions unanswered. Someone reported a much greater slowdown than Diego measured, so this must be investigated until we can find the cause. -- M?ns Rullg?rd mans at mansr.com
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines).
What evidence is there to support this claim?
Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again.
if i remove some unneeded code and that results in a 0.5% gain on one machine chances are it also does on most others, its not as if the removial of code will likely make it slower.
You can expect that removal of useless code won't make things slower _on average_. However if the CPU use of the removed code was significantly less than 0.5% of the total then you can also expect the consistent effect to be masked by random variation - something like 50.1% of machines showing better benchmark results and 49.9% showing worse.
Hi, Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines). What evidence is there to support this claim?
Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again.
Could we please see the code ?
if i remove some unneeded code and that results in a 0.5% gain on one machine chances are it also does on most others, its not as if the removial of code will likely make it slower.
You can expect that removal of useless code won't make things slower _on average_. However if the CPU use of the removed code was significantly less than 0.5%
??? Code was _useless_ so CPU did never _use_ it. [...] -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA checking for life_signs in -lkenny... no
On Tue, 2008-12-16 at 10:55 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines). What evidence is there to support this claim?
Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again.
Could we please see the code ?
I deleted it already - it was just random stuff to add code size. But you can get equivalent effects in a few tries. Just add a global (so it's not optimized away) function in the middle of h264.c and write random stuff there.
if i remove some unneeded code and that results in a 0.5% gain on one machine chances are it also does on most others, its not as if the removial of code will likely make it slower.
You can expect that removal of useless code won't make things slower _on average_. However if the CPU use of the removed code was significantly less than 0.5%
??? Code was _useless_ so CPU did never _use_ it.
By "useless" I mean both unused code and code which is executed but makes no difference to the computation result.
On Tue, Dec 16, 2008 at 09:09:55PM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 10:55 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines). What evidence is there to support this claim?
Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again.
Could we please see the code ?
I deleted it already - it was just random stuff to add code size. But you can get equivalent effects in a few tries. Just add a global (so it's not optimized away) function in the middle of h264.c and write random stuff there.
What iam wondering is how a speed effect from random unused code on a single machine is related to your claim above of lack of correlations between the speed on different machines. One actual case here, that is the table split showed a slowdown consistant across machines, thus contradicting your claim. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I know you won't believe me, but the highest form of Human Excellence is to question oneself and others. -- Socrates
On Tue, 2008-12-16 at 20:45 +0100, Michael Niedermayer wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines). What evidence is there to support this claim?
Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again.
What iam wondering is how a speed effect from random unused code on a single machine is related to your claim above of lack of correlations between the speed on different machines.
Do you really expect adding an unused function to consistently increase speed everywhere? (BTW the quoted claim above comes from Panagiotis Issaris, not me like you claimed).
One actual case here, that is the table split showed a slowdown consistant across machines, thus contradicting your claim.
Why would it contradict my claim (even if verified to be consistent, I'm not sure whether there were enough reports for that yet)? I myself reported a 8% slowdown which is more than the ordinary random variation. Of course you'll see consistent change _direction_ if there's a big enough real effect on speed. What should be rare according to my claim is a nontrivial(*) change having a consistent small negative or consistent small positive effect across machines. (*)mainly meant to exclude changes which have no effect on code generation or memory layout - something like changing the number of iterations in a loop could have a consistent effect
On Tue, Dec 16, 2008 at 10:54:35PM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 20:45 +0100, Michael Niedermayer wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote: > Besides that, in my opinion you can't benchmark code on one particular > machine and expect a 0.5% performance loss to say anything about the > codes performance on other machines (other then that the code hasn't > introduced a substantial performance change on similar machines). What evidence is there to support this claim?
Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again.
What iam wondering is how a speed effect from random unused code on a single machine is related to your claim above of lack of correlations between the speed on different machines.
Do you really expect adding an unused function to consistently increase speed everywhere?
Please refrain from randomly cliping parts of the discussion silently, and by that placing what i said under some different text. Also please refrain from putting words in my mouth, or replying in such a way as to pretend that I made some ridiculous claim that you now would try to correct. Also please refrain from trolling in the style of: U: > Its always A, thats fact, known by everyone M: no its not always A M: > no its not always A U: Do you really belive its never A?
(BTW the quoted claim above comes from Panagiotis Issaris, not me like you claimed).
One actual case here, that is the table split showed a slowdown consistant across machines, thus contradicting your claim.
Why would it contradict my claim (even if verified to be consistent, I'm not sure whether there were enough reports for that yet)? I myself reported a 8% slowdown which is more than the ordinary random variation. Of course you'll see consistent change _direction_ if there's a big enough real effect on speed.
What should be rare according to my claim is a nontrivial(*) change having a consistent small negative or consistent small positive effect across machines.
well the split seems to have a rather consistant effect across machines. that is its slower. And peoples reactions, like diego posting a IRC log from you shows that you made people belive that your claims where related to the actual code changes. If OTOH you really are just talking about hypothetical changes that have not happened (or that you dont know or will not name) then i really must ask you to leave as that would be off topic. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I know you won't believe me, but the highest form of Human Excellence is to question oneself and others. -- Socrates
On Tue, 2008-12-16 at 22:43 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 10:54:35PM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 20:45 +0100, Michael Niedermayer wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote: > On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote: >> Besides that, in my opinion you can't benchmark code on one particular >> machine and expect a 0.5% performance loss to say anything about the >> codes performance on other machines (other then that the code hasn't >> introduced a substantial performance change on similar machines). > What evidence is there to support this claim?
Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again.
What iam wondering is how a speed effect from random unused code on a single machine is related to your claim above of lack of correlations between the speed on different machines.
Do you really expect adding an unused function to consistently increase speed everywhere?
Please refrain from randomly cliping parts of the discussion silently, and by that placing what i said under some different text.
The only part I clipped was Baptiste asking about the particular code. Do you think that affected the context?
Also please refrain from putting words in my mouth, or replying in such a way as to pretend that I made some ridiculous claim that you now would try to correct.
I don't think I did.
Also please refrain from trolling in the style of: U: > Its always A, thats fact, known by everyone M: no its not always A
M: > no its not always A U: Do you really belive its never A?
I did not do that either. There was no such "always" / "not never" difference above. Only "changes are consistent" or "are not". You: How is it known that small speed changes are not consistent across machines? Me: You can see that the speed is affected by random effects like this even on a single machine. You: How does that show all the machines do not behave the same? Me: Once we know there's a random component it's unlikely all the machines would get the same random result isn't it? Did you interpret the original claim to say that even _major_ performance differences would not correlate between different enough machines? I don't think that was meant, and I certainly didn't interpret it that way. If you see a small change in a benchmark result then you can't say much about what'll happen on other machines, except that the changes are unlikely to be too big on similar enough machines (there's more chance they could vary significantly on substantially different machines, but there's still correlation).
One actual case here, that is the table split showed a slowdown consistant across machines, thus contradicting your claim.
Why would it contradict my claim (even if verified to be consistent, I'm not sure whether there were enough reports for that yet)? I myself reported a 8% slowdown which is more than the ordinary random variation. Of course you'll see consistent change _direction_ if there's a big enough real effect on speed.
What should be rare according to my claim is a nontrivial(*) change having a consistent small negative or consistent small positive effect across machines.
well the split seems to have a rather consistant effect across machines. that is its slower. And peoples reactions, like diego posting a IRC log from you shows that you made people belive that your claims where related to the actual code changes.
What I said on IRC was that a 0.28% change in the h264 decoder tells essentially nothing. A 0.28% slowdown in a benchmark is not a much bigger sign of real slowdown than a 0.28% benchmark speedup would be. So my claims there were about the interpretation of that one benchmark result. After hearing of Diego's -0.28% result alone there was little reason to expect the speed change to be consistently up or to be consistently down. Only after knowing of my -8% result too was there evidence of some effect that might make results consistently slower.
On Wed, Dec 17, 2008 at 01:39:57AM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 22:43 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 10:54:35PM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 20:45 +0100, Michael Niedermayer wrote:
Uoti Urpala wrote: > On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote: >> On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote: [...] You: How is it known that small speed changes are not consistent across machines?
i did not ask this. You claimed that small speed changes behave completely random across machines. I pointed out that you had no evidence for this claim and where presenting your wild guesses as facts. this has mislead ffmpeg developers in beliving these claims and could damage the quality of the project, thus i strongly suggest you back your claims up or say that you are sorry for making such baseless claims.
Me: You can see that the speed is affected by random effects like this even on a single machine.
You: How does that show all the machines do not behave the same?
I did not ask this first, because the way its worded would implicate that you did show a random effect on a single machine, you did _NOT_ show this to begin with. You _claimed_ to have found that _DIFFERENT_ code was executing at different speed. This is not disputed though strictly you even refused to provide any details about what exactly you tested. how that now relates to a random effect on a single machine is entirely in your head it seems, the following step:
Me: Once we know there's a random component it's unlikely all the machines would get the same random result isn't it?
its not hard to proof anything based on a false assumtation. [...]
One actual case here, that is the table split showed a slowdown consistant across machines, thus contradicting your claim.
Why would it contradict my claim (even if verified to be consistent, I'm not sure whether there were enough reports for that yet)? I myself reported a 8% slowdown which is more than the ordinary random variation. Of course you'll see consistent change _direction_ if there's a big enough real effect on speed.
What should be rare according to my claim is a nontrivial(*) change having a consistent small negative or consistent small positive effect across machines.
well the split seems to have a rather consistant effect across machines. that is its slower. And peoples reactions, like diego posting a IRC log from you shows that you made people belive that your claims where related to the actual code changes.
What I said on IRC was that a 0.28% change in the h264 decoder tells essentially nothing. A 0.28% slowdown in a benchmark is not a much bigger sign of real slowdown than a 0.28% benchmark speedup would be. So my claims there were about the interpretation of that one benchmark result. After hearing of Diego's -0.28% result alone there was little reason to expect the speed change to be consistently up or to be consistently down. Only after knowing of my -8% result too was there evidence of some effect that might make results consistently slower.
An interrestingly convoluted way of saying you were totally wrong [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Why not whip the teacher when the pupil misbehaves? -- Diogenes of Sinope
As much as I loathe to get into this pointless discussion... On Wed, Dec 17, 2008 at 01:15:29AM +0100, Michael Niedermayer wrote:
On Wed, Dec 17, 2008 at 01:39:57AM +0200, Uoti Urpala wrote:
What I said on IRC was that a 0.28% change in the h264 decoder tells essentially nothing. A 0.28% slowdown in a benchmark is not a much bigger sign of real slowdown than a 0.28% benchmark speedup would be. So my claims there were about the interpretation of that one benchmark result. After hearing of Diego's -0.28% result alone there was little reason to expect the speed change to be consistently up or to be consistently down. Only after knowing of my -8% result too was there evidence of some effect that might make results consistently slower.
An interrestingly convoluted way of saying you were totally wrong
No, it is just an interestingly convoluted way of saying that different facts lead to different conclusions. And now can you guys *please* just drop it? I can use all the help I can get in trying to make heads or tails of this. It is puzzling all the assembler gurus why shifting a table around has different effects in different places, you can imagine I hardly know where to start looking for answers... Diego
On Wed, Dec 17, 2008 at 01:35:33AM +0100, Diego Biurrun wrote: [...]
I can use all the help I can get in trying to make heads or tails of this. It is puzzling all the assembler gurus why shifting a table around has different effects in different places, you can imagine I hardly know where to start looking for answers...
one thing is scan8[0] gcc can with a static const table replace this by a constant but not with a non static one in a seperate file. I do not know if there are more such tables, and have not verified if this is the only issue. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Those who are too smart to engage in politics are punished by being governed by those who are dumber. -- Plato
On Wed, 2008-12-17 at 01:15 +0100, Michael Niedermayer wrote:
On Wed, Dec 17, 2008 at 01:39:57AM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 22:43 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 10:54:35PM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 20:45 +0100, Michael Niedermayer wrote:
> Uoti Urpala wrote: > > On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote: [...] You: How is it known that small speed changes are not consistent across machines?
i did not ask this.
I think that is an OK way to paraphrase the discussion around your "What evidence..." to Panagiotis Issaris. It's obviously not meant to be an exact quote.
You claimed that small speed changes behave completely random across machines.
Essentially yes.
I pointed out that you had no evidence for this claim and where presenting your wild guesses as facts.
False. The claim was based on the fact that you can see how "random" effects affect the performance even on individual machines, and the changes are triggered by differences smaller than the typical differences between machines. That does not require any "wild guesses".
this has mislead ffmpeg developers in beliving these claims and could damage the quality of the project, thus i strongly suggest you back your claims up or say that you are sorry for making such baseless claims.
I think I have already explained the reasoning. You can see that irrelevant changes randomly affect the performance of a binary, and also combine with the additional effect of applying a patch so that a patch can be "harmful" but after an irrelevant change elsewhere adding it can be "beneficial" or the other way around. I have personally tested that to some degree with the h264 decoder in particular on two different systems.
Me: You can see that the speed is affected by random effects like this even on a single machine.
You: How does that show all the machines do not behave the same?
I did not ask this first, because the way its worded would implicate that you did show a random effect on a single machine, you did _NOT_ show this to begin
Does your claim "not show" mean anything else than that there's no way I could strictly speaking "prove" I really ran the test I said I had run?
with. You _claimed_ to have found that _DIFFERENT_ code was executing at different speed. This is not disputed though strictly you even refused to provide any details about what exactly you tested.
Actually the example I gave was adding completely unused code to h264.c.
how that now relates to a random effect on a single machine is entirely in your head it seems, the following step:
It shows that the performance of the binary depends to some degree on small details such as the existence of unused code somewhere (and exact length of rarely used code, etc). That means it has an essentially random component since you can't control all the relevant variables with enough precision.
Me: Once we know there's a random component it's unlikely all the machines would get the same random result isn't it?
its not hard to proof anything based on a false assumtation.
What exactly are you calling false? Are you saying that the performance of the h264 decoder has no noticeable dependence on small details that are going to differ between almost every version and machine?
What I said on IRC was that a 0.28% change in the h264 decoder tells essentially nothing. A 0.28% slowdown in a benchmark is not a much
An interrestingly convoluted way of saying you were totally wrong
I still don't think a 0.28% benchmark change tells anything significant. So what I said earlier was not wrong, and I did not say it was.
Uoti Urpala wrote:
On Tue, 2008-12-16 at 10:55 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines). What evidence is there to support this claim? Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again. Could we please see the code ?
I deleted it already - it was just random stuff to add code size. But you can get equivalent effects in a few tries. Just add a global (so it's not optimized away) function in the middle of h264.c and write random stuff there.
Well, considering the "random" factor is discussed here, I won't take words for granted. If you want to make your point valid, give a code example.
if i remove some unneeded code and that results in a 0.5% gain on one machine chances are it also does on most others, its not as if the removial of code will likely make it slower. You can expect that removal of useless code won't make things slower _on average_. However if the CPU use of the removed code was significantly less than 0.5% ??? Code was _useless_ so CPU did never _use_ it.
By "useless" I mean both unused code and code which is executed but makes no difference to the computation result.
I hope and assume this is not the case with current code, so your point is void. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA checking for life_signs in -lkenny... no
On Tue, 2008-12-16 at 11:48 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 10:55 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote:
Besides that, in my opinion you can't benchmark code on one particular machine and expect a 0.5% performance loss to say anything about the codes performance on other machines (other then that the code hasn't introduced a substantial performance change on similar machines). What evidence is there to support this claim? Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again. Could we please see the code ?
I deleted it already - it was just random stuff to add code size. But you can get equivalent effects in a few tries. Just add a global (so it's not optimized away) function in the middle of h264.c and write random stuff there.
Well, considering the "random" factor is discussed here, I won't take words for granted. If you want to make your point valid, give a code example.
How would giving a code example affect its validity? Did my description of the change not give enough information about it? It's not like you could yourself verify that the particular example I used behaves the way I said - it's unlikely the random effects would be the same on your system. If you want to test the random effects yourself I gave one way to do that above.
if i remove some unneeded code and that results in a 0.5% gain on one machine chances are it also does on most others, its not as if the removial of code will likely make it slower. You can expect that removal of useless code won't make things slower _on average_. However if the CPU use of the removed code was significantly less than 0.5% ??? Code was _useless_ so CPU did never _use_ it.
By "useless" I mean both unused code and code which is executed but makes no difference to the computation result.
I hope and assume this is not the case with current code, so your point is void.
Your comment makes no sense whatsoever. What are you trying to say? You're quoting a part where you seem to confuse "useless" with "unused" and I clarify what I meant with "useless". From this you jump to something completely different. What "your point"? Something other than you quoted? My main point in the thread has been that there are random changes which mask small real speed changes in benchmarks, but you're not saying anything sensible related to that either.
Uoti Urpala wrote:
On Tue, 2008-12-16 at 11:48 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:31 +0100, Michael Niedermayer wrote:
On Tue, Dec 16, 2008 at 11:01:34AM +0100, Panagiotis Issaris wrote: > Besides that, in my opinion you can't benchmark code on > one particular machine and expect a 0.5% performance loss > to say anything about the codes performance on other > machines (other then that the code hasn't introduced a > substantial performance change on similar machines). What evidence is there to support this claim? Many known cases where changes that are known not to affect the real quality of the code alter performance by more than 0.5%. I just tried adding an unused global function to h264.c. The first attempt didn't show any quickly measurable difference. Then I made the function a bit larger, and now the benchmark ran consistently 0.8% faster. Removing the unused function made things correspondingly 0.8% slower again. Could we please see the code ? I deleted it already - it was just random stuff to add code size. But you can get equivalent effects in a few tries. Just add a global (so it's not optimized away) function in the middle of
On Tue, 2008-12-16 at 10:55 -0800, Baptiste Coudurier wrote: h264.c and write random stuff there. Well, considering the "random" factor is discussed here, I won't take words for granted. If you want to make your point valid, give a code example.
How would giving a code example affect its validity?
You know, what is called "proof".
Did my description of the change not give enough information about it? It's not like you could yourself verify that the particular example I used behaves the way I said - it's unlikely the random effects would be the same on your system.
If you want to test the random effects yourself I gave one way to do that above.
What are you trying to say here ? That no benchmark ever can be trusted, because it has everything has random effects on every system ?
if i remove some unneeded code and that results in a 0.5% gain on one machine chances are it also does on most others, its not as if the removial of code will likely make it slower. You can expect that removal of useless code won't make things slower _on average_. However if the CPU use of the removed code was significantly less than 0.5% ??? Code was _useless_ so CPU did never _use_ it. By "useless" I mean both unused code and code which is executed but makes no difference to the computation result. I hope and assume this is not the case with current code, so your point is void.
Your comment makes no sense whatsoever. What are you trying to say?
Im saying that your point is void because the current code is assumed to not contain anything useless, and adding "useless" code would be stupid, but I thought mentioning this would be useless as this is obvious. [...] -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA checking for life_signs in -lkenny... no
On Tue, 2008-12-16 at 12:32 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 11:48 -0800, Baptiste Coudurier wrote:
Well, considering the "random" factor is discussed here, I won't take words for granted. If you want to make your point valid, give a code example.
How would giving a code example affect its validity?
You know, what is called "proof".
What would it prove? I already gave the results. You can't reproduce the same ones so it's pointless to give the exact same code. But you should be able to reproduce the existence of random variation with the method I gave.
Did my description of the change not give enough information about it? It's not like you could yourself verify that the particular example I used behaves the way I said - it's unlikely the random effects would be the same on your system.
If you want to test the random effects yourself I gave one way to do that above.
What are you trying to say here ?
That you can't reproduce the same random values I did. If you want to verify the existence of random variation you need to find the samples that show differences on your system.
That no benchmark ever can be trusted, because it has everything has random effects on every system ?
That has nothing to do with what I was saying in the part I quoted. But what I've said elsewhere in the thread is basically that small benchmark changes show little else than the absence of very significant changes either way.
> if i remove some unneeded code and that results in a 0.5% > gain on one machine chances are it also does on most > others, its not as if the removial of code will likely make > it slower. You can expect that removal of useless code won't make things slower _on average_. However if the CPU use of the removed code was significantly less than 0.5% ??? Code was _useless_ so CPU did never _use_ it. By "useless" I mean both unused code and code which is executed but makes no difference to the computation result. I hope and assume this is not the case with current code, so your point is void.
Your comment makes no sense whatsoever. What are you trying to say?
Im saying that your point is void because the current code is assumed to
Which "your point"? I don't see how your claim would make any sense if interpreted to be about my main point in this thread.
not contain anything useless, and adding "useless" code would be stupid, but I thought mentioning this would be useless as this is obvious.
Your comments here are such complete nonsense it's hard to even tell what mistake you're making and correct it...
Uoti Urpala wrote:
On Tue, 2008-12-16 at 12:32 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 11:48 -0800, Baptiste Coudurier wrote:
Well, considering the "random" factor is discussed here, I won't take words for granted. If you want to make your point valid, give a code example. How would giving a code example affect its validity? You know, what is called "proof".
What would it prove? I already gave the results.
No, you threw numbers saying something you cannot prove, and you assume we should trust your words. This is stupid.
You can't reproduce the same ones so it's pointless to give the exact same code. But you should be able to reproduce the existence of random variation with the method I gave.
If I cannot reproduce, your point is null and your claim is absurd.
Did my description of the change not give enough information about it? It's not like you could yourself verify that the particular example I used behaves the way I said - it's unlikely the random effects would be the same on your system.
If you want to test the random effects yourself I gave one way to do that above. What are you trying to say here ?
That you can't reproduce the same random values I did. If you want to verify the existence of random variation you need to find the samples that show differences on your system.
This is pure absurdity.
That no benchmark ever can be trusted, because it has everything has random effects on every system ?
That has nothing to do with what I was saying in the part I quoted.
Let's go forward, will you ? This has everything to do with what you said. You claim that since every change produce random effects on every computer, no benchmark is worth, no proof is worth being showed. If you want to say anything, please give valid arguments. This is all absurd and stupid IMHO. [...]
>> if i remove some unneeded code and that results in a >> 0.5% gain on one machine chances are it also does on >> most others, its not as if the removial of code will >> likely make it slower. > You can expect that removal of useless code won't make > things slower _on average_. However if the CPU use of the > removed code was significantly less than 0.5% ??? Code was _useless_ so CPU did never _use_ it. By "useless" I mean both unused code and code which is executed but makes no difference to the computation result. I hope and assume this is not the case with current code, so your point is void. Your comment makes no sense whatsoever. What are you trying to say? Im saying that your point is void because the current code is assumed to
Which "your point"?
I won't repeat myself. You are intelligent to understand and no playing around, I hope.
[...]
not contain anything useless, and adding "useless" code would be stupid, but I thought mentioning this would be useless as this is obvious.
Your comments here are such complete nonsense it's hard to even tell what mistake you're making and correct it...
Blabla, playing around won't make your point valid, I say your point is void, because code is assumed to not contain anything useless, so nothing useless will _ever_ be added, so don't even consider it nor talk about it. Is it so hard to understand for you ? Debating with you is really tiring since you make so many efforts to show that you do not understand what people say while you obviously do. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA checking for life_signs in -lkenny... no
On Tue, 2008-12-16 at 13:10 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 12:32 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 11:48 -0800, Baptiste Coudurier wrote:
Well, considering the "random" factor is discussed here, I won't take words for granted. If you want to make your point valid, give a code example. How would giving a code example affect its validity? You know, what is called "proof".
What would it prove? I already gave the results.
No, you threw numbers saying something you cannot prove, and you assume we should trust your words. This is stupid.
I told you the facts, and I told you how to verify them if you want to. What's stupid is your argumentation in this thread.
You can't reproduce the same ones so it's pointless to give the exact same code. But you should be able to reproduce the existence of random variation with the method I gave.
If I cannot reproduce, your point is null and your claim is absurd.
I just told you how to reproduce the existence of random effects. And there are lots of other facts that you're _not_ capable of reproducing and which still are not "absurd".
Did my description of the change not give enough information about it? It's not like you could yourself verify that the particular example I used behaves the way I said - it's unlikely the random effects would be the same on your system.
If you want to test the random effects yourself I gave one way to do that above. What are you trying to say here ?
That you can't reproduce the same random values I did. If you want to verify the existence of random variation you need to find the samples that show differences on your system.
This is pure absurdity.
Why? You couldn't find any stupider and less constructive way to argue?
That no benchmark ever can be trusted, because it has everything has random effects on every system ?
That has nothing to do with what I was saying in the part I quoted.
Let's go forward, will you ? This has everything to do with what you said.
You quoted some text and asked whether that was what I was trying to say "here". So I answered that it was not. And I added this part: "But what I've said elsewhere in the thread is basically that small benchmark changes show little else than the absence of very significant changes either way.".
You claim that since every change produce random effects on every computer, no benchmark is worth, no proof is worth being showed.
You snipped the part reproduced above where I clearly say something else and then wrote this false claim?
If you want to say anything, please give valid arguments.
This is all absurd and stupid IMHO.
I think you should take your own advice. Try to give some actual arguments instead of repeating "absurd". Your inability to form a coherent argument is what makes this stupid.
Your comment makes no sense whatsoever. What are you trying to say? Im saying that your point is void because the current code is assumed to
Which "your point"?
I won't repeat myself. You are intelligent to understand and no playing around, I hope.
I in turn am not certain whether the content of your post is due to your relatively low intelligence or if you're intentionally behaving stupidly.
not contain anything useless, and adding "useless" code would be stupid, but I thought mentioning this would be useless as this is obvious.
Your comments here are such complete nonsense it's hard to even tell what mistake you're making and correct it...
Blabla, playing around won't make your point valid, I say your point is void, because code is assumed to not contain anything useless, so nothing useless will _ever_ be added, so don't even consider it nor talk about it. Is it so hard to understand for you ?
That adding unused code affects performance in essentially random ways means that changes to meaningful code will also cause similar "random" performance changes as a side effect.
Debating with you is really tiring since you make so many efforts to show that you do not understand what people say while you obviously do.
I honestly thought that you'd understand the point above. Sorry if I did not adequately consider the limits of your intelligence.
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:10 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 12:32 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 11:48 -0800, Baptiste Coudurier wrote:
Well, considering the "random" factor is discussed here, I won't take words for granted. If you want to make your point valid, give a code example. How would giving a code example affect its validity? You know, what is called "proof". What would it prove? I already gave the results. No, you threw numbers saying something you cannot prove, and you assume we should trust your words. This is stupid.
I told you the facts, and I told you how to verify them if you want to. What's stupid is your argumentation in this thread.
You are even contradicting yourself because you claimed that changes and symptoms were random, how can you guarantee that my tests would verify them ? This is still absurd, and your argumentation is still stupid.
You can't reproduce the same ones so it's pointless to give the exact same code. But you should be able to reproduce the existence of random variation with the method I gave. If I cannot reproduce, your point is null and your claim is absurd.
I just told you how to reproduce the existence of random effects. And there are lots of other facts that you're _not_ capable of reproducing and which still are not "absurd".
This is still absurd, see the previous point.
Did my description of the change not give enough information about it? It's not like you could yourself verify that the particular example I used behaves the way I said - it's unlikely the random effects would be the same on your system.
If you want to test the random effects yourself I gave one way to do that above. What are you trying to say here ? That you can't reproduce the same random values I did. If you want to verify the existence of random variation you need to find the samples that show differences on your system. This is pure absurdity.
Why? You couldn't find any stupider and less constructive way to argue?
What is pure absurdity is claiming that I cannot reproduce the same random values that you did, since you told me what to do to experience the same random values .... You are even reexplaining how to experience it. Now please backup your claims and stop this "randomness" pointless discussion.
That no benchmark ever can be trusted, because it has everything has random effects on every system ? That has nothing to do with what I was saying in the part I quoted. Let's go forward, will you ? This has everything to do with what you said.
You quoted some text and asked whether that was what I was trying to say "here". So I answered that it was not. And I added this part: "But what I've said elsewhere in the thread is basically that small benchmark changes show little else than the absence of very significant changes either way.".
You claim that since every change produce random effects on every computer, no benchmark is worth, no proof is worth being showed.
You snipped the part reproduced above where I clearly say something else and then wrote this false claim?
Twisting words as you always do won't lead you somewhere this time. You deliberatly claimed that showing proof wouldn't makes your point valid: "How would giving a code example affect its validity?" But you keep deliberatly quoting only what you want...
If you want to say anything, please give valid arguments.
This is all absurd and stupid IMHO.
I think you should take your own advice. Try to give some actual arguments instead of repeating "absurd". Your inability to form a coherent argument is what makes this stupid.
You inability to remember your own claims, once again show how absurd your whole argumentation is.
Your comment makes no sense whatsoever. What are you trying to say? Im saying that your point is void because the current code is assumed to Which "your point"? I won't repeat myself. You are intelligent to understand and no playing around, I hope.
I in turn am not certain whether the content of your post is due to your relatively low intelligence or if you're intentionally behaving stupidly.
Insulting people when you out of valid arguments can only reflect how stuborn you are.
not contain anything useless, and adding "useless" code would be stupid, but I thought mentioning this would be useless as this is obvious. Your comments here are such complete nonsense it's hard to even tell what mistake you're making and correct it... Blabla, playing around won't make your point valid, I say your point is void, because code is assumed to not contain anything useless, so nothing useless will _ever_ be added, so don't even consider it nor talk about it. Is it so hard to understand for you ?
That adding unused code affects performance in essentially random ways means that changes to meaningful code will also cause similar "random" performance changes as a side effect.
Your point is void until you show proof that adding "useless" code cause slowdown _AND_ speedup. Stop claming things without backup.
Debating with you is really tiring since you make so many efforts to show that you do not understand what people say while you obviously do.
I honestly thought that you'd understand the point above. Sorry if I did not adequately consider the limits of your intelligence.
Your lack of understanding is not an excuse for insulting people. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA checking for life_signs in -lkenny... no
On Tue, 2008-12-16 at 14:29 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:10 -0800, Baptiste Coudurier wrote:
No, you threw numbers saying something you cannot prove, and you assume we should trust your words. This is stupid.
I told you the facts, and I told you how to verify them if you want to. What's stupid is your argumentation in this thread.
You are even contradicting yourself because you claimed that changes and symptoms were random, how can you guarantee that my tests would verify them ? This is still absurd, and your argumentation is still stupid.
I say a thrown die can end up any side up, and present a sequence of throws which shows every number appearing. You cannot reproduce the exact sequence of throws but you can reproduce every number appearing. You asking for the particular code changes I used is like asking for the particular way I threw the die to produce a certain number.
What are you trying to say here ? That you can't reproduce the same random values I did. If you want to verify the existence of random variation you need to find the samples that show differences on your system. This is pure absurdity.
Why? You couldn't find any stupider and less constructive way to argue?
What is pure absurdity is claiming that I cannot reproduce the same random values that you did, since you told me what to do to experience the same random values ....
I told you you can't reproduce the exact same random values but you can reproduce the same overall behavior. You can't reproduce the exact die throws but you can reproduce every side coming up at some point. What's so hard to understand about this?
You quoted some text and asked whether that was what I was trying to say "here". So I answered that it was not. And I added this part: "But what I've said elsewhere in the thread is basically that small benchmark changes show little else than the absence of very significant changes either way.".
You claim that since every change produce random effects on every computer, no benchmark is worth, no proof is worth being showed.
You snipped the part reproduced above where I clearly say something else and then wrote this false claim?
Twisting words as you always do won't lead you somewhere this time. You deliberatly claimed that showing proof wouldn't makes your point valid:
"How would giving a code example affect its validity?"
Now how is your reply related to the part you're replying to? One talks about how benchmark results should be interpreted, the other about reproducing test results. And no the part you quoted last does not say anything about what showing proof would or wouldn't do. It says that giving the particular code I used in my test would not be proof because it would not help you reproduce the exact _same_ random effects, unless you somehow cloned a lot more of my system.
not contain anything useless, and adding "useless" code would be stupid, but I thought mentioning this would be useless as this is obvious. Your comments here are such complete nonsense it's hard to even tell what mistake you're making and correct it... Blabla, playing around won't make your point valid, I say your point is void, because code is assumed to not contain anything useless, so nothing useless will _ever_ be added, so don't even consider it nor talk about it. Is it so hard to understand for you ?
That adding unused code affects performance in essentially random ways means that changes to meaningful code will also cause similar "random" performance changes as a side effect.
Your point is void until you show proof that adding "useless" code cause slowdown _AND_ speedup. Stop claming things without backup.
I mentioned a case where adding unused code gave a 0.8% speedup. I think it should be rather obvious that it's not always consistently beneficial.
Debating with you is really tiring since you make so many efforts to show that you do not understand what people say while you obviously do.
I honestly thought that you'd understand the point above. Sorry if I did not adequately consider the limits of your intelligence.
Your lack of understanding is not an excuse for insulting people.
Do you claim your own tone was not insulting? Maybe I failed to understand your failure to understand the issue being discussed, but I think that's a lesser fault than yours.
Uoti Urpala wrote:
On Tue, 2008-12-16 at 14:29 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
On Tue, 2008-12-16 at 13:10 -0800, Baptiste Coudurier wrote:
No, you threw numbers saying something you cannot prove, and you assume we should trust your words. This is stupid. I told you the facts, and I told you how to verify them if you want to. What's stupid is your argumentation in this thread. You are even contradicting yourself because you claimed that changes and symptoms were random, how can you guarantee that my tests would verify them ? This is still absurd, and your argumentation is still stupid.
I say a thrown die can end up any side up, and present a sequence of throws which shows every number appearing. You cannot reproduce the exact sequence of throws but you can reproduce every number appearing. You asking for the particular code changes I used is like asking for the particular way I threw the die to produce a certain number.
Comparing the situation to "die throwing" is awkward IMHO, considering your first sentence, you need to show me first that your die can end up and down. I did see this claim about the code, however Im still waiting for your backup, until this I do not trust you, and I refute your claim. Furthermore code relates in nothing to die, since a die has a fixed number of faces at the beginning, code has probably unlimited number of variants, especially in this case (H264). Please choose a better example. This might explain why you fail to understand why you argumentation is absurd.
What are you trying to say here ? That you can't reproduce the same random values I did. If you want to verify the existence of random variation you need to find the samples that show differences on your system. This is pure absurdity. Why? You couldn't find any stupider and less constructive way to argue? What is pure absurdity is claiming that I cannot reproduce the same random values that you did, since you told me what to do to experience the same random values ....
I told you you can't reproduce the exact same random values but you can reproduce the same overall behavior. You can't reproduce the exact die throws but you can reproduce every side coming up at some point. What's so hard to understand about this?
First, see previous point, then after that, considering I accept your "die throwing" comparision, while I don't, I want to know how you threw the "die", and without this I consider your claim and everything you say pointless. What's so hard to understand about this ? That's all about benchmark.
You quoted some text and asked whether that was what I was trying to say "here". So I answered that it was not. And I added this part: "But what I've said elsewhere in the thread is basically that small benchmark changes show little else than the absence of very significant changes either way.".
You claim that since every change produce random effects on every computer, no benchmark is worth, no proof is worth being showed. You snipped the part reproduced above where I clearly say something else and then wrote this false claim? Twisting words as you always do won't lead you somewhere this time. You deliberatly claimed that showing proof wouldn't makes your point valid:
"How would giving a code example affect its validity?"
Now how is your reply related to the part you're replying to? One talks about how benchmark results should be interpreted, the other about reproducing test results.
I'm talking about your interpretation of benchmarking _and_ about reproducing test results. In this regard, what I said earlier totally makes sense.
And no the part you quoted last does not say anything about what showing proof would or wouldn't do.
Of course it does backup the fact that you said that showing your test case would not affect the validity of your point, and I say this is false, in case this was not obvious to you.
It says that giving the particular code I used in my test would not be proof because it would not help you reproduce the exact _same_ random effects, unless you somehow cloned a lot more of my system.
Again throwing random claims and random numbers without backup is pointless. Please backup your claims once again.
not contain anything useless, and adding "useless" code would be stupid, but I thought mentioning this would be useless as this is obvious. Your comments here are such complete nonsense it's hard to even tell what mistake you're making and correct it... Blabla, playing around won't make your point valid, I say your point is void, because code is assumed to not contain anything useless, so nothing useless will _ever_ be added, so don't even consider it nor talk about it. Is it so hard to understand for you ? That adding unused code affects performance in essentially random ways means that changes to meaningful code will also cause similar "random" performance changes as a side effect. Your point is void until you show proof that adding "useless" code cause slowdown _AND_ speedup. Stop claming things without backup.
I mentioned a case where adding unused code gave a 0.8% speedup. I think it should be rather obvious that it's not always consistently beneficial.
Again: "Your point is void until you show proof that adding "useless" code cause slowdown _AND_ speedup. Stop claming things without backup." Im still waiting for your claim backup.
Debating with you is really tiring since you make so many efforts to show that you do not understand what people say while you obviously do. I honestly thought that you'd understand the point above. Sorry if I did not adequately consider the limits of your intelligence.
Your lack of understanding is not an excuse for insulting people.
Do you claim your own tone was not insulting?
It was definitely not, again please backup your claims.
Maybe I failed to understand your failure to understand the issue being discussed, but I think that's a lesser fault than yours.
You still fail to understand why your argumentation is absurd, nonetheless that really does not excuse your insulting tone. Now please stop this pointless and futile discussion. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA checking for life_signs in -lkenny... no
On Tue, 2008-12-16 at 15:31 -0800, Baptiste Coudurier wrote:
Uoti Urpala wrote:
I say a thrown die can end up any side up, and present a sequence of throws which shows every number appearing. You cannot reproduce the exact sequence of throws but you can reproduce every number appearing. You asking for the particular code changes I used is like asking for the particular way I threw the die to produce a certain number.
Comparing the situation to "die throwing" is awkward IMHO, considering your first sentence, you need to show me first that your die can end up and down. I did see this claim about the code, however Im still waiting for your backup, until this I do not trust you, and I refute your claim.
Furthermore code relates in nothing to die, since a die has a fixed number of faces at the beginning, code has probably unlimited number of variants, especially in this case (H264). Please choose a better example.
I told you how to reproduce the random behavior. Pick some h264 video and benchmark decoder performance with that. Then add an unused function in the middle of h264.c and repeat the benchmark with the new binary. If there was no change then make the function bigger (it probably doesn't make much difference exactly what it computes) and repeat. So there is a clearly defined way to see performance variations caused by even unused code. I've told you how to test it yourself if you don't want to take my word for it. Then why are still complaining? I can't tell exactly which changes will cause what performance variations on your system, but I'd expect you to hit some before too long. Why can't you just do that test instead of complaining that I gave no proof?
On Wed, Dec 17, 2008 at 01:05:12AM +0200, Uoti Urpala wrote:
On Tue, 2008-12-16 at 14:29 -0800, Baptiste Coudurier wrote: [ .. flames and insults .. ]
Please stop it here, the only direction this subthread is going is down. Diego
On Sun, Dec 14, 2008 at 03:58:26PM -0800, Jason Garrett-Glaser wrote:
no, the apparent slowdown of cathedral-beta2-400extra-crop-avc.mp4 is not ok
Why is this not ok, while disabling filter_mb_fast was OK? I measure a 25% performance increase on a sample here by re-enabling filter_mb_fast. Why is 0.5% here intolerable, but 25% there is just fine?
stop whining, ive reenabled it though only if CODEC_FLAG2_FAST is set. If you want it enabled without this, patch to fix it or limit it to cases where its spec compliant is welcome. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The educated differ from the uneducated as much as the living from the dead. -- Aristotle
participants (10)
-
baptiste.coudurier@gmail.com -
cehoyos@ag.or.at -
darkshikari@gmail.com -
diego@biurrun.de -
mans@mansr.com -
michaelni@gmx.at -
robert.swain@gmail.com -
takis.issaris@uhasselt.be -
tempn@twmi.rr.com -
uoti.urpala@pp1.inet.fi