[FFmpeg-devel] [PATCH] add signature filter for MPEG7 video signature

[Thilo Borgmann]

Am 21.03.16 um 00:14 schrieb Gerion Entrup:
> On Sonntag, 20. März 2016 17:01:17 CET Thilo Borgmann wrote:
>>> On Sun, Mar 20, 2016 at 12:00:13PM +0100, Gerion Entrup wrote:
>> [...]
>>
>>> This filter does not implement all features of MPEG7. Missing features:
>>>
>>> - binary output
>>> - compression of signature files
>>
>> I assume these features are optional?
> Compression is optional (could be set as flag in the binary representation). I
> have not found, whether binary output is optional. 

> It is definitely possible to only work with the XML-Files.

Of course, but having an unspecified XML output is almost useless if binary
output is not optional. So I think it is crucial to know what the spec says
about output.


>>> - work only on (cropped) parts of the video
>>
>> How useful is this then? Has fingerprint computed only on (cropped) parts of
>> the video any value outside of FFmpeg itself - does this comply to the spec
>> so that it can be compared with any other software generating it?
> To clarify, the filter does not crop anything. The standard defines an optional 
> cropping to, I guess, concentrate on specific video parts (this is not 
> implemented). Assuming someone is recording a monitor, then e.g. the unrelated 
> part of the video could be cropped out. Beside that, the signature itself is 
> invariant to cropping up to a certain limit.
> 
> The cropping values (upper left and bottom right position are specified in the 
> xml, so another software could either crop the same way or compare only with 
> the cropped input.
> (The fact, that ffmpeg has a cropping filter, would make such a feature some 
> kind of redundant.)

If I understand it correctly, the filter should not crop the image but only use
the pixel information within the specified area or the whole image. Making it a
filter option is useful, because the fingerprint of a part of the image can be
used in a filter chain continuing with the entire image (no actual crop is
required).


> The XML is standard compliant. 

So XML output is compliant to the spec? Or is the XML itself just valid XML?


> The signature is not bitexact. 3-4 (ternary) 
> values in the framesignature differ from the signature of the sample files, but 
> the conformence tests [1] allow up to 15 ternaryerrors. 

Bitexact compared to what?
Does it allow up to 15 ternary errors for assume two inputs are equal enough to
be the same image or does it state that the fingerprint itself may differ for 15
ternary errors for the very same image?


> I'm not exactly sure 
> why the difference occur, I guess, this are floating point issues, because my 
> implementation calculates the values with a slightly different method than the 
> reference software. Currently, I don't have the sample files anymore. If there 
> is interest, I will try to get them again.
> 
> [1] https://www.itscj.ipsj.or.jp/sc29/open/29view/29n11696t.doc

Being compliant to the reference output is crucial, otherwise we calculate just
some fingerprint but not the MPEG7 fingerprint... don't you think?

-Thilo