[Ffmpeg-devel] [PATCH] fix (?) 2pass convergence errors

[Loren Merritt]

On Mon, 4 Sep 2006, Corey Hickey wrote:
> Loren Merritt wrote:
>> On Mon, Sep 04, 2006 at 12:35:21PM -0700, Corey Hickey wrote:
>>> According to several tests, the average of qscale more or less matches
>>> the average qp of the first pass and the average of new_qscale sort of
>>> matches the average qp of the second pass. Both of them get thrown off
>>> when lmin is below qmin. Is there a better way? I understand
>>> approximately 5% of the ratecontrol code so far, so I'm probably
>>> overlooking something easy.
>>
>> for(...)
>>      qp_sum += clip(rcc->entry[i].new_qscale, qmin, qmax);
>> qp_sum /= num_entries * FF_QP2LAMBDA;
>
> Ahh. I get your meaning, I think. I changed my patch to use your
> suggestion, with the modification that I divide rcc->entry[i] by
> FF_QP2LAMBDA before clipping it.
>
>> This is still not perfect, because having lambda<qp does increase bitrate
>> a little.
>
> Yes, the result seems somewhat low; for instance, it reports 2.something
> when the eventual average qp is 4.something (unless I did something wrong).
>
> If it's not feasible to calculate at this point, I'm not averse to
> dropping that part of my patch. I only tried it because Michael
> mentioned it, but it's not required.

IMO, setting lmin<qmin is just a bad idea anyway, so there's no need to 
optimize for that case. e.g. encoding a frame with qp=4,lambda=2.0 is 
worse quality per bitrate than qp=3,lambda=3.0.

qmin<lmin might be useful in combination with qprd, and is harmless otherwise.

--Loren Merritt