[Ffmpeg-devel] Sample Aspect Ratio & Pan&Scan

Måns Rullgård mru
Mon Nov 6 20:07:46 CET 2006


Steve Lhomme <slhomme at divxcorp.com> writes:

> M?ns Rullg?rd wrote:
>> Steve Lhomme <slhomme at divxcorp.com> writes:
>>
>>> Hi,
>>>
>>> So far in DrFFMPEG we used a modified version of mpeg12.c that doesn't
>>> take the pan_scan size to set the sample_aspect_ratio. (see around
>>> line 2142) But since FFMPEG doesn't seem to have problems to deal with
>>> it I think we may do something wrong...
>>>
>>> Because of DivX certification constraints we only output square
>>> pixels. So we have to translate the source aspect ratio to square
>>> pixels.
>> That paragraph doesn't make sense.
>
> It just means that in what we're doing, we handle the aspect ratio
> before doing the encoding, and we always code square pixels (PAR=1:1).
>
>>> So far we assumed that the sample_aspect_ratio was a pixel aspect
>>> ratio and it worked well for most format. But it seems that it's not
>>> good enough in the general case.
>> sample_aspect_ratio is just that, the aspect ratio of each pixel.
>
> So SAR = PAR ?

Yes.

> So we were right so far... But then that means FFMPEG is not
> consistent/clean when dealing with pan&scan.

The problem is that there is no clear spec for what to do.

>>> For example I have 2 VOB files. They both use pan & scan and have a
>>> pixel aspect ratio of 16:9. But for the first one (chems1.vob) we get
>>> a sample aspect ratio of 256:135 (1.896) and the other one has a
>>> sample aspect ratio of 32/27 (1.18). For chems1.vob the original pixel
>>> size is 720x576 and for the other one it's 720x480. While the source
>>> dimensions and pixel aspect ratio are similar, the sample aspect ratio
>>> is very different. That's because the pan & scan dimensions are very
>>> different (for chems1.vob the width is bigger than the height).
>> The meanings of display_horizontal_size and display_vertical_size are
>> not defined in the MPEG2 spec.  The fields are there merely as hints
>> that the display system may use in whatever way it sees fit.
>> Looking at chems1.vob (assuming it is the same file I've gotten from
>> you previously), it has a DAR of 16/9 (not SAR as you say above) with
>> a coded picture size is 720x576.  The display_horizontal_size and
>> display_vertical_size values are 540x576.  One possible interpretation
>> of these values (which is suggested in the MPEG2 spec) is that the
>> display device has a pixel size of 540x576 and a display aspect ratio
>> of 16/9.  That would give a pixel aspect ratio of 5/3 or 1.67.
>> Another reasonable interpretation is that the DAR of 16/9 refers to
>> the full coded picture, which would give a SAR of 64/27 or 2.37.  It
>> is not obvious from the video image what the intent is.
>
> 2.37 is wrong. It would give an upscale dimension of 576*2.37=1365 and
> the actual dimension to display is around 1024 pixels which is
> according to MPC (displaying the image correctly, unlike VLC) a DAR of
> 1.78 (16:9).

My bad.  I don't know what I did to get those numbers.  Applying the
DAR to the full picture gives a SAR of 64/45=1.42.  Applying the DAR
to just the display size gives a SAR of 256/135=1.89.

>>> Our hack so far was not to take the pan & scan in account and it
>>> worked well. So is there any reason we should use the pan&scan
>>> value ? From what I understand the pan&scan feature is made to crop
>>> the
>>> original picture ? On demand ? Maybe it should be optional in
>>> FFMPEG ?
>> FFmpeg doesn't process pan&scan information other than exporting it
>> to the calling application.  This is as it should be.
>
> Well, this pan&scan information is exported in AVFrame->pan_scan,
> but as it's supposed to be about cropping it should rather be in
> AVCodecContext or something like that.

The pan&scan vectors can change per frame, and frequently do, so they
must be in AVFrame.

> Plus the resulting dimensions of the pan_scan values are already
> applied sample_aspect_ratio. And if
> AVCodecContext->sample_aspect_ratio is supposed to be the PAR then
> the value is wrong.
>
>>> It seems that encoding with FFMPEG works because both the (pixel)
>>> aspect_ratio_info is stored and the sample_aspect_ratio. But the
>>> aspect_ratio_info field is MpegEncContext specific... Should it be
>>> more general ?
>> The aspect_ratio_info field in MpegEncContext is the raw encoding of
>> aspect ratio used in the bitstream.  Its meaning depends on the codec,
>> and exporting it would make no sense.
>
> Actually it's the pan_scan info that should be more general. Why is it
> in AVFrame and not in AVCodecContext ? because an MPEG stream can have
> various values in the same stream ? If so the upper app is supposed to
> change the cropping value during decoding too ?
>
> So to summarize what there is now in the code:
>
> final DAR = sample_aspect_ratio * width / height
> width and height being the pixels after the pan_scan is applied. But
> the app doesn't have any info on the pan_scan apart from when a frame
> is decoded.

The size of the pan&scan region is constant, but the position changes.

-- 
M?ns Rullg?rd
mru at inprovide.com




More information about the ffmpeg-devel mailing list