[FFmpeg-devel] [PATCH] avfilter: add panorama filter

[James Almer]

On 3/9/2018 12:12 PM, James Almer wrote:
> On 3/9/2018 12:04 PM, Paul B Mahol wrote:
>> On 3/9/18, James Almer <jamrial at gmail.com> wrote:
>>> On 3/9/2018 7:05 AM, Paul B Mahol wrote:
>>>> On 3/9/18, Paul B Mahol <onemda at gmail.com> wrote:
>>>>> On 3/9/18, wm4 <nfxjfg at googlemail.com> wrote:
>>>>>> On Fri, 9 Mar 2018 09:15:13 +0100
>>>>>> Paul B Mahol <onemda at gmail.com> wrote:
>>>>>>
>>>>>>> On 3/9/18, wm4 <nfxjfg at googlemail.com> wrote:
>>>>>>>> On Thu, 8 Mar 2018 21:53:48 -0300
>>>>>>>> James Almer <jamrial at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> On 3/8/2018 9:50 PM, Hazem Ashmawy wrote:
>>>>>>>>>> [PATCH] avfilter: add panorama filter
>>>>>>>>>>
>>>>>>>>>> Sorry about that! I removed them now.
>>>>>>>>>> For the future, any recommendation for a tool for linting /
>>>>>>>>>> checking
>>>>>>>>>> formating
>>>>>>>>>> rules?
>>>>>>>>>
>>>>>>>>> There's tools/patcheck. Feed it a git format-patch style of patch to
>>>>>>>>> find common issues, but keep in mind it can generate a lot of false
>>>>>>>>> positives.
>>>>>>>>>
>>>>>>>>> I don't know if we have documentation about actual formatting rules
>>>>>>>>> anywhere.
>>>>>>>>
>>>>>>>> Also:
>>>>>>>>
>>>>>>>> <_jamrial> shouldn't that panorama filter sent to the ml use the
>>>>>>>> spherical
>>>>>>>> frame side data?
>>>>>>>>
>>>>>>>> I think so.
>>>>>>>
>>>>>>> Are there actual files that have such data?
>>>>>>
>>>>>> Is that a trick question? I only know the non-standard, Google specific
>>>>>> metadata in mkv and mp4 that lavf can read (was any of this
>>>>>> standardized yet?).
>>>>>>
>>>>>> But that doesn't change that we can tag AVFrames with this info, and
>>>>>> for files which don't have the metadata, it makes sense to me to set it
>>>>>> with a new vf_format argument or some sort of vf_setinfo (if we don't
>>>>>> have anything like this yet).
>>>>>>
>>>>>> The part that is annoying is that vf_panorama still seems to require
>>>>>> setting an output projection, which would make the whole thing more
>>>>>> annoying instead of less, but even then I'd argue it should default to
>>>>>> taking the AVFrame configuration (AV_FRAME_DATA_SPHERICAL) as input by
>>>>>> default, even if the filter arguments can override it.
>>>>>
>>>>> That frame side data is very specific and thus considered barely useful
>>>>> here.
>>>>>
>>>>> Is it at all updated to the latest improvements, like new equi-angular
>>>>> cubemap projection?
>>>>>
>>>>
>>>> I guess not at all, I get this:
>>>>
>>>> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x21c1740] Unknown projection type
>>>
>>> Sample? Also, where is this new projection defined?
>>
>> https://www.youtube.com/watch?v=qhLExhpXX0E
>>
>> It is defined by Google?
> 
> Vittorio and I used
> https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md
> to write the current Matroska and mov implementations, and by extension
> the AVSphericalMapping API. Specifically the Equirectangular and Cubemap
> projections.
> 
> This sample seems to have a "ytmp" projection box, but it's not defined
> in the above document. I guess the name hints at it being a very early
> an internal draft? We can't really do much without a spec...

The h264/mp4 version has this unknown "ytmp" box, but the vp9/webm one
has a ProjectionType element with a value of 3, plus some binary data in
the ProjectionPrivate element, which according to the spec means a Mesh
projection. We don't currently support that one.

Weird, seeing mp4 supposedly has the "mshp" box for this.

> 
> In any case, I insist much like wm4 that a filter like this should use
> the metadata stored in the AVFrame if available.
>