[FFmpeg-devel] [FFmpeg-devel-irc] IRC log for 2010-03-01

[Jason Garrett-Glaser]

On Tue, Mar 2, 2010 at 5:31 PM, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Tue, Mar 02, 2010 at 12:00:05AM +0000, irc at mansr.com wrote:
> [...]
>> [17:24:54] <Dark_Shikari> gthreepwood: oh, so it isn't a block-matching motion estimation project
>> [17:24:56] * BBB kicks mru
>> [17:25:08] <Dark_Shikari> you're writing a ROTATION-aware motion estimation algorithm
>> [17:25:12] <Dark_Shikari> now _THAT_ is actually interesting
>> [17:25:35] * BBB thinks Dark_Shikari isn't paying attention when he hears buzzwords
>> [17:25:45] <Dark_Shikari> Now, you will have to find some way to signal the rotation amount in the bitstream, and you'll need to find a way to fill the holes created by rotation (e.g. on the corners)
>> [17:26:10] <Dark_Shikari> and you will need to find a fast but good enough rotation interpolation algorithm so that the decoder doesn't get 5 times slower
>> [17:26:12] <gthreepwood> Yes, all interesting problems :)
>> [17:26:21] <Dark_Shikari> I've already designed a way to signal it in the bitstream though
>> [17:26:22] <Dark_Shikari> it's trivial
>> [17:26:43] <Dark_Shikari> with CABAC, here's how I'd do it:
>> [17:26:53] <Dark_Shikari> 1) We start by assuming 0 degree rotation.
>> [17:26:53] <mru> if pixels were triangular, we could use hexagonal blocks and rotations by pi/3 would leave no holes
>> [17:27:22] <Dark_Shikari> 2) We get a bit. ?If the bit is 0, we stop, no rotation. ?If 1, we continue.
>> [17:27:38] <Dark_Shikari> 3) We get a bit. ?If the bit is 0, we rotate by -90. ?If the bit is 1, we rotate by +90.
>> [17:27:45] <Dark_Shikari> repeat, each time refining the rotation
>> [17:27:50] <Dark_Shikari> so the next step, it'll go +/- 45
>> [17:27:54] <Dark_Shikari> then +/- 22.5
>
> i doubt this is efficent, most rotations likely would be close to 0 and
> likely quite spatially correlated, id try to code and predict them like a 3rd
> component of MVs. And you can make scaling blocks the 4th.

As discussed later (might not have been in the same channel), I
suggested rotation prediction, i.e. instead of coding absolute
rotations, code deltas, and do it like mvd coding in CABAC.

You're right that such a method is probably not very efficient, at
second thought.

>> [17:27:57] <gthreepwood> I see, that's an interesting approach
>> [17:28:05] <Dark_Shikari> thus the encoder can choose an arbitrary level of precision based on whatever is RD-optimal
>> [17:28:13] <Dark_Shikari> and it simply writes an EOB bit at the end when it's done
>> [17:28:20] <kshishkov> mru: "always study either pie-in-the-sky or long-obsolete stuff, ?sometimes both
>> [17:28:22] <Dark_Shikari> and thanks to arithmetic coding, it's cheap.
>> [17:28:26] <Dark_Shikari> Relatively. ?We hope. ?Maybe.
>> [17:28:36] <kshishkov> mru: sounds like "academic study" definition
>> [17:28:53] <Dark_Shikari> Though intuitively it seems as though it cannot actually code 180 degree rotations
>> [17:28:58] <Dark_Shikari> thanks, Zeno's paradox
>> [17:29:26] <gthreepwood> so you think I should be looking at modifying x264? I thought it might be a bit complex, although I haven't yet looked at the code
>> [17:29:30] <Dark_Shikari> but anyways, if you wanted to do this in x264, you would need the following:
>> [17:29:45] <Dark_Shikari> 1) Modify the motion compensation functions (x264_mb_mc and relatives) in common/macroblock.c
>> [17:30:00] <Dark_Shikari> 2) Find some place in the macroblock cache to store the rotation of each block (h->mb.cache.rotation or something?)
>> [17:30:09] <Dark_Shikari> I'd say that one rotation syntax element per partition is reasonable
>> [17:30:17] <Dark_Shikari> so an 8x16 block would have two rotation syntax elements.
>> [17:30:55] <Dark_Shikari> 3) Decide at what point you're going to do the rotation analysis and modify the motion search to consider it.
>> [17:31:05] <Dark_Shikari> 4) Modify ffh264 decoder to do the same thing
>
>> [17:31:54] <kshishkov> Dark_Shikari: does all that sounds to me only that "iterative systems" is the next logical step ;)
>> [17:31:54] <Dark_Shikari> the primary reason I recommend x264, other than bias, is that nobody in this channel really understands any of the mpegvideo encoders in ffmpeg, especially not the motion estimation parts ;)
>> [17:33:06] <gthreepwood> So the x264 code, that's in the separate x264 project, right?
>> [17:33:21] <Dark_Shikari> yes
>> [17:33:37] <gthreepwood> Thanks for your helpful comments
>> [17:33:40] <kshishkov> decoder for it is still in FFmpeg though
>> [17:34:01] <BBB> Dark_Shikari, nobody understands the decoder either :)
>
> hello, my name is nobody ;)
>
> gthreepwood, (me hopes he reads this) you could try the rotate in snow
> because, if its good in terms of quality vs bitrate and speed we can actually
> add it to the spec easily. Also snow uses mpegvideo/motion_est so you would
> be able to test a wavelet codec (snow) and mpeg4-asp with minimal additional
> code

Very good point, the rotation makes a whole lot of sense for Snow's
approach, though it might be a bit tricky to reconcile it with OBMC
effectively.

Dark Shikari