Hi On Thu, Dec 29, 2005 at 12:25:03AM +0100, Michael Niedermayer wrote:

Hi

On Wed, Dec 28, 2005 at 12:35:09PM -0800, Corey Hickey wrote:

...

Michael Niedermayer CVS wrote:

...

Update of /cvsroot/ffmpeg/ffmpeg/libavcodec In directory mail:/var2/tmp/cvs-serv19133

Modified Files: motion_est.c Log Message: hmm did i mess the scaling of these up at some point? anyway, this change decreases bitrate and increase PSNR by ~1.00 on my test file, other files also benefit significantly

I tested this on The Matrix:

----------------------------------------------------------------- for i in 1:turbo 2 ; do time mencoder ~/dumpstream/matrix.vob -aid 128 -oac copy \ -vf crop=718:356:0:60,scale=640:272 -sws 9 -ovc lavc -lavcopts \ vcodec=mpeg4:vbitrate=581:psnr:vpass=$i:mbd=2:mv0:trell:cbp:\ precmp=2:cmp=2:subcmp=2:predia=2:dia=2:preme=2:vme=5:v4mv:\ last_pred=2:vqcomp=0.6:vmax_b_frames=2:qpel -ofps 24000/1001 \ -o test.avi done

----------------------------------------------------------------- before:

(pass 1) PSNR: Y:40.06, Cb:44.34, Cr:44.64, All:41.10 user 35m12.394s

(pass 2) PSNR: Y:42.10, Cb:45.17, Cr:45.85, All:42.96 user 186m52.325s

----------------------------------------------------------------- after:

(pass 1) PSNR: Y:40.05, Cb:44.29, Cr:44.59, All:41.08

(pass 2) PSNR: Y:42.04, Cb:45.15, Cr:45.84, All:42.91

-----------------------------------------------------------------

I don't have the user times listed for "after" because for some reason my CPU frequency scaling went screwy and stuck my CPU at a low frequency. Nevertheless, I don't think the encoding speeds are significantly different -- the user times for after the next commit were 34m52.509s and 190m8.100s, respectively for each pass.

Unfortunately, I didn't see a large PSNR gain like I had hoped from the log message. The PSNR actually dropped a bit. As for quality differences, here's the breakdown from looking at several different scenes. - mostly the same (low-motion scenes are all the same) - several places that look slightly better - a few places that look slightly worse - a few places that look significantly better - one place that looks significantly worse - one frame that looks way worse (flickering lightning for a second)

Overall, I have the impression that quality is improved.

good, i also havnt seen such huge PSNR improvments on all files, +1.00 was just the maximum IIRC, ill post the exact PSNR from my files as soon as the system on which they are is up again (might take some time ...), i should have posted them already when i did the testing anyway its subjective quality that counts not PSNR

btw, if you want you could try to replace all 'c->scene_change_score+= s->qscale' in motion_est.c with 'c->scene_change_score+= s->qscale*N' where N is a interger between 1..16 (16 should be the old, 1 the new) according to my tests 1 was best overall on my files but the differences at the low values where small (all IIRC as i cant check that ATM), so maybe we should increase it ... you could also try different values for scene_change_threshold ... allthough 6h encoding for each case maybe a serious problem, my files where all short <=2min [...] -- Michael

Corey Hickey wrote:

Michael Niedermayer wrote:

...

btw, if you want you could try to replace all 'c->scene_change_score+= s->qscale' in motion_est.c with 'c->scene_change_score+= s->qscale*N' where N is a interger between 1..16 (16 should be the old, 1 the new) according to my tests 1 was best overall on my files but the differences at the low values where small (all IIRC as i cant check that ATM), so maybe we should increase it ... you could also try different values for scene_change_threshold ... allthough 6h encoding for each case maybe a serious problem, my files where all short <=2min

Well, I have a script running now that will test the various values in this order: 4 8 12 16 2 6 10 14 1 3 5 7 9 11 13 15

If and when I see a trend I can kill the script and narrow it down.

I modified libavcodec and mplayer to make the N you mention be configurable and ran my tests. The speed impact is undetectable in my current setup. Here are the PSNRs so far: ------------------------------------------------------------------- sc_factor=1 PSNR: Y:42.04, Cb:45.15, Cr:45.84, All:42.91 sc_factor=2 PSNR: Y:42.04, Cb:45.15, Cr:45.82, All:42.91 sc_factor=3 in progress... sc_factor=4 PSNR: Y:42.06, Cb:45.15, Cr:45.84, All:42.93 sc_factor=5 not done sc_factor=6 PSNR: Y:42.08, Cb:45.16, Cr:45.85, All:42.95 sc_factor=7 not done sc_factor=8 PSNR: Y:42.08, Cb:45.16, Cr:45.84, All:42.95 sc_factor=9 not done sc_factor=10 PSNR: Y:42.09, Cb:45.16, Cr:45.85, All:42.96 sc_factor=11 not done sc_factor=12 PSNR: Y:42.09, Cb:45.16, Cr:45.84, All:42.96 sc_factor=13 not done sc_factor=14 PSNR: Y:42.09, Cb:45.16, Cr:45.84, All:42.96 sc_factor=15 not done sc_factor=16 PSNR: Y:42.09, Cb:45.16, Cr:45.84, All:42.96 ------------------------------------------------------------------- 1 (the current default), 2, and 4 look pretty much the same. At 14 and 16 I see some high-motion scenes that benefit from receiving an I-frame, but other places that have high motion but not enough to trigger an I-frame look worse. At 10, 12, and 8 (to a lesser extent), I can't find the scenes that get another I-frame, and other high-motion scenes still look worse. 6 looks nice. I can't find anywhere that looks worse, and the high-motion scenes with extra I-frames still look much better. I don't know how universally applicable these tests are, but if 16 was default before and 1 is default now, adjusting to somewhere toward the middle is probably a safe change. I've attached patches for ffmpeg and mplayer, if anyone else wants to test this. If the patches should be applied, I'll make a patch for the mplayer man page. Otherwise, 6 could just be hardcoded. I haven't tried sc_threshold yet. My understanding is that using sc_factor differs from sc_threshold in that sc_factor makes frames with high quantizers have a greater impact in triggering an I-frame. Is that correct? -Corey

Michael Niedermayer wrote:

...

I don't know how universally applicable these tests are, but if 16 was default before and 1 is default now, adjusting to somewhere toward the middle is probably a safe change. I've attached patches for ffmpeg and mplayer, if anyone else wants to test this. If the patches should be applied, I'll make a patch for the mplayer man page. Otherwise, 6 could just be hardcoded.

I haven't tried sc_threshold yet. My understanding is that using

can you achive the same improvement with sc_threshold? if yes the patch is rejected, if no the patch is ok and can be applied

Well, that took a really long time, mostly because I didn't have any idea what values to start with. I have failed to achieve the same quality improvement with sc_threshold. Here are the PSNR results: sc_threshold=0: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-10: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-20: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-30: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-100: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-200: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-300: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-400: PSNR: Y:42.06, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-500: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-600: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-700: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-800: PSNR: Y:42.06, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-900: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-1000: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-1100: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-1200: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-1300: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-1400: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-1500: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-2000: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-2500: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.95 sc_threshold=-3000: PSNR: Y:42.08, Cb:45.18, Cr:45.89, All:42.96 sc_threshold=-3500: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.95 sc_threshold=-4000: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-4500: PSNR: Y:42.08, Cb:45.18, Cr:45.89, All:42.96 sc_threshold=-5000: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.95 sc_threshold=-5500: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-6000: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-7000: PSNR: Y:42.09, Cb:45.18, Cr:45.87, All:42.96 sc_threshold=-8000: PSNR: Y:42.09, Cb:45.18, Cr:45.87, All:42.96 sc_threshold=-9000: PSNR: Y:42.09, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-10000: PSNR: Y:42.09, Cb:45.18, Cr:45.87, All:42.96 Or, if you prefer, a graph is attached. As you can see, the PSNR hits a plateau at around sc_threshold=-3000, so I don't think testing lower values would yield any better PSNR. As far as visual quality goes, I didn't see any of the same gains as with my sc_factor patch. I found a few new I-frames at higher values of sc_threshold, but they weren't placed in locations that improved bad-looking high-motion scenes as much as with sc_factor=6 (which, incidentally, yields 42.97 PSNR with current CVS). I've attached patches for ffmpeg and mplayer; they're like the last ones I posted, except: * I made 6 the default * I added sc_factor to the mplayer man page * for convenience, I moved all sc_factor lines to right after sc_threshold You approved my previous patches contingent upon the improvement not being attainable with sc_threshold; are these ones ok as well? Also, I suppose making 6 the default will change the regression tests. I would include them in the patch, but "make test" fails for me right now regardless. It might be easier if you apply the ffmpeg part yourself. -Corey

Michael Niedermayer wrote:

...

As you can see, the PSNR hits a plateau at around sc_threshold=-3000, so I don't think testing lower values would yield any better PSNR. As far

maybe try -20000, -40000 and -80000 too just to be sure

Ok, I'm doing so now.

...

as visual quality goes, I didn't see any of the same gains as with my sc_factor patch. I found a few new I-frames at higher values of sc_threshold, but they weren't placed in locations that improved bad-looking high-motion scenes as much as with sc_factor=6 (which, incidentally, yields 42.97 PSNR with current CVS).

iam curious how vqcomp=0.7 or larger and various vi_qfactors (both negative and positive) would affect these relations :)

Well, I'm sure they would.... Now that I think about it, higher values of vqcomp might work best with higher sc_factor values.

...

+{"sc_factor", NULL, OFFSET(scenechange_factor), FF_OPT_TYPE_INT, 6, 1, 16, V|E},

why 16? larger values should be ok too

That was the range you suggested I test, originally. I suppose I might as well make it INT_MAX and note in the man page that 1-16 is a sane range.

except these the patch is ok and can be applied

Does this pair of patches look ok? I want to be sure I'm not about to break something on my first ffmpeg commit. :) Differences: * sc_factor=1 is default * INT_MAX is the maximum value * all sc_factor lines are at the bottom, below other parameters * mplayer man page description is a bit more detailed -Corey

Michael Niedermayer wrote:

...

...

except these the patch is ok and can be applied

Does this pair of patches look ok? I want to be sure I'm not about to break something on my first ffmpeg commit. :) Differences:

* sc_factor=1 is default * INT_MAX is the maximum value * all sc_factor lines are at the bottom, below other parameters * mplayer man page description is a bit more detailed

[...]

...

+ int scenechange_factor; + + /** * minimum lagrange multipler * - encoding: set by user. * - decoding: unused

ok, strictly this doesnt break ABI compatibility due to AVOptions but i really would prefer if it where at the end (and iam sure everyone who doesnt use AVOptions would be happier too)

Sorry, I missed that one. This should be (finally) correct. Since I think you're ok with them otherwise, I'll apply both patches when I wake up tomorrow unless someone says "stop". Thanks, -Corey

Michael Niedermayer wrote:

...

...

...

I don't know how universally applicable these tests are, but if 16 was default before and 1 is default now, adjusting to somewhere toward the middle is probably a safe change. I've attached patches for ffmpeg and mplayer, if anyone else wants to test this. If the patches should be applied, I'll make a patch for the mplayer man page. Otherwise, 6 could just be hardcoded.

I haven't tried sc_threshold yet. My understanding is that using

can you achive the same improvement with sc_threshold? if yes the patch is rejected, if no the patch is ok and can be applied

Well, that took a really long time, mostly because I didn't have any idea what values to start with. I have failed to achieve the same quality improvement with sc_threshold. Here are the PSNR results:

sc_threshold=0: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-10: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-20: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-30: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-100: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-200: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-300: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-400: PSNR: Y:42.06, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-500: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-600: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.93 sc_threshold=-700: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-800: PSNR: Y:42.06, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-900: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-1000: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-1100: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-1200: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-1300: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-1400: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-1500: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.94 sc_threshold=-2000: PSNR: Y:42.06, Cb:45.17, Cr:45.86, All:42.94 sc_threshold=-2500: PSNR: Y:42.07, Cb:45.17, Cr:45.87, All:42.95 sc_threshold=-3000: PSNR: Y:42.08, Cb:45.18, Cr:45.89, All:42.96 sc_threshold=-3500: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.95 sc_threshold=-4000: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-4500: PSNR: Y:42.08, Cb:45.18, Cr:45.89, All:42.96 sc_threshold=-5000: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.95 sc_threshold=-5500: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-6000: PSNR: Y:42.08, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-7000: PSNR: Y:42.09, Cb:45.18, Cr:45.87, All:42.96 sc_threshold=-8000: PSNR: Y:42.09, Cb:45.18, Cr:45.87, All:42.96 sc_threshold=-9000: PSNR: Y:42.09, Cb:45.18, Cr:45.88, All:42.96 sc_threshold=-10000: PSNR: Y:42.09, Cb:45.18, Cr:45.87, All:42.96

Or, if you prefer, a graph is attached.

As you can see, the PSNR hits a plateau at around sc_threshold=-3000, so I don't think testing lower values would yield any better PSNR. As far

maybe try -20000, -40000 and -80000 too just to be sure

Append to the list above: sc_threshold=-12000: PSNR: Y:42.08, Cb:45.17, Cr:45.86, All:42.95 sc_threshold=-14000: PSNR: Y:42.07, Cb:45.16, Cr:45.85, All:42.95 sc_threshold=-16000: PSNR: Y:42.07, Cb:45.16, Cr:45.85, All:42.94 sc_threshold=-18000: PSNR: Y:42.06, Cb:45.15, Cr:45.82, All:42.93 sc_threshold=-20000: PSNR: Y:42.05, Cb:45.14, Cr:45.81, All:42.92 sc_threshold=-40000: PSNR: Y:41.57, Cb:44.82, Cr:45.39, All:42.46 sc_threshold=-80000: PSNR: Y:39.07, Cb:43.38, Cr:43.62, All:40.11 It looks like last time I stopped testing right before the PSNR decreases. The encodes made at sc_threshold=-14000 and below actually end up with more I-frames than sc_factor=6 -- but they don't look as good. Also, PSNR-wise, sc_factor=6 does better than any sc_threshold value I found: sc_factor=6: PSNR: Y:42.10, Cb:45.17, Cr:45.87, All:42.97 -Corey