On Mon, 15 Oct 2007 14:11:02 +0200, Loren Merritt <lorenm at u.washington.edu> wrote:
On Mon, 15 Oct 2007, Martin Zlomek wrote:
I have discovered that h264_loop_filter_strength_mmx2(), used for 'bS' calculation in inter predicted macroblocks, gives wrong output when used in fields (compared with the reference decoder). Attached patch is temporary not-optimized solution giving correct output.
I would like to ask Loren Merritt for help - could you look at this, please? Learning MMX instructions and understanding your code would not take me a while...
It's probably due to mvy_limit. I don't know how to fix it without sacrificing some speed (not that I benchmarked anything), so I would just leave filter_mb_fast for the common case that needs to be fast (i.e. progressive), and let PAFF uses the non-asm code. But if you really want: patch is untested and doesn't modify the prototype and caller to match.
--Loren Merritt
After deeper analysis, I think that the problem is not in the MMX routine. Different output is given on several test samples when using non-asm code as well. The problem is probably in indeces into h->non_zero_count_cache or in h->non_zero_count_cache as such - this array is used in the condition when assigning bS[i] value of 2: for( i = 0; i < 4; i++ ) { int x = dir == 0 ? edge : i; int y = dir == 0 ? i : edge; int b_idx= 8 + 4 + x + 8*y; int bn_idx= b_idx - (dir ? 8:1); if( h->non_zero_count_cache[b_idx] != 0 || h->non_zero_count_cache[bn_idx] != 0 ) { bS[i] = 2; } else if(!mv_done) { Working on exact problem identification and on patch... -- Martin Zlomek martin.zlomek at email.cz