[FFmpeg-devel] [PATCH] Implement PAFF in H.264

[Martin Zlomek]

On Mon, 15 Oct 2007 14:11:02 +0200, Loren Merritt  
<lorenm at u.washington.edu> wrote:

> On Mon, 15 Oct 2007, Martin Zlomek wrote:
>
>> I have discovered that h264_loop_filter_strength_mmx2(), used for
>> 'bS' calculation in inter predicted macroblocks, gives wrong output
>> when used in fields (compared with the reference decoder).
>> Attached patch is temporary not-optimized solution giving correct
>> output.
>>
>> I would like to ask Loren Merritt for help - could you look at this,
>> please? Learning MMX instructions and understanding your code would
>> not take me a while...
>
> It's probably due to mvy_limit. I don't know how to fix it without
> sacrificing some speed (not that I benchmarked anything), so I would just
> leave filter_mb_fast for the common case that needs to be fast (i.e.
> progressive), and let PAFF uses the non-asm code.
> But if you really want: patch is untested and doesn't modify the
> prototype and caller to match.
>
> --Loren Merritt


After deeper analysis, I think that the problem is not in the MMX routine.
Different output is given on several test samples when using non-asm code
as well.
The problem is probably in indeces into h->non_zero_count_cache or in
h->non_zero_count_cache as such - this array is used in the condition when
assigning bS[i] value of 2:

                 for( i = 0; i < 4; i++ ) {
                     int x = dir == 0 ? edge : i;
                     int y = dir == 0 ? i    : edge;
                     int b_idx= 8 + 4 + x + 8*y;
                     int bn_idx= b_idx - (dir ? 8:1);

                     if( h->non_zero_count_cache[b_idx] != 0 ||
                         h->non_zero_count_cache[bn_idx] != 0 ) {
                         bS[i] = 2;
                     }
                     else if(!mv_done)
                     {


Working on exact problem identification and on patch...


-- 
Martin Zlomek
martin.zlomek at email.cz