[FFmpeg-devel] [PATCH v2] avcodec/jpeg2000dec: support of 2 fields in 1 AVPacket

[Tomas Härdin]

ons 2024-02-21 klockan 15:27 +0100 skrev Jerome Martinez:
> On 21/02/2024 14:11, Tomas Härdin wrote:
> 
> > mxfdec can detect cases where there will be two separate fields in
> > one
> > KLV,
> 
> In practice the issue is not to detect I2 case in mxfdec (it saves us
> only a little during probing of the first frame, but I could add such
> signaling in a patch after this one because it is actually
> independent 
> from the management of 2 fields in the decoder if we want to support 
> buggy files), the issue is to split per field in mxfdec.
> 
> SMPTE ST 422 extracts:
> "Users are cautioned that the code values for SOC and EOC are not 
> protected and can occur within the image size marker segment (SIZ), 
> quantization marker segment (QCD), comment marker segment (COM) and 
> other places."
> "Decoders can derive the bytestream offsets of each field by
> analysing 
> the code stream format within the essence element as described in 
> ISO/IEC 15444-1."
> 
> Note that this MXF + jp2k spec hints that separating fields should be
> done in the decoder, not the demuxer.

We already have a j2k parser that could be pressed into service for
this. But perhaps parsing is not necessary, see below. My main concern
is that interlacing work the same for all codecs that mxfdec supports,
expecially rawvideo.

> It is impossible to split per field in a codec-neutral manner due to 
> lack of metadata for that in MXF, and doing that in mxfdec implies to
> duplicate jp2k header parser code in mxfdec, and to add a similar
> parser 
> per supported video format in the future.

We do have a bit of duplicate parsing in mxfdec (and mxfenc) already,
which I don't like. So I share your concern.

> > and the decoder(s) can if I'm not mistaken be instructed to decode
> > into an AVFrame with stride and offset set up for interlaced
> > decoding.
> 
> 
> I checked the MPEG-2 Video decoder and it does not do what you say

I didn't say that it does, I said that we could do that. If we conform
all wrappings to MIXED_FIELDS this is probably fine. I think it's been
well established that FFmpeg is MIXED_FIELDS internally.

> , it does what I do with this patch:
> - mpegvideo_parser (so the parser of raw files, equivalent to mxfdec)
> understands that the stream has 2 separate fields and puts them in 1 
> single AVPacket. It could separate them put it doesn't.

mpeg2 has a concept of fields that j2k doesn't. Again, we're not really
talking about j2k here but MXF


> > It should be possible to have ffmpeg set up the necessary plumbing
> > for
> > this.
> 
> But is it how it works elsewhere in FFmpeg? Would such complex and
> deep 
> modifications be accepted by others?

Good question. I would propose something like the following:

1) detect the use of SEPARATE_FIELDS and set a flag in AVStream
2) allocate AVFrame for the size of the resulting *frame*
3a) if the codec is inherently interlaced, call the decoder once
3b) if the codec is not inherently interlaced, call the decoder twice,
with appropriate stride, and keep track of the number of bytes decoded
so far so we know what offset to start the second decode from

The codecs for which 3b) applies include at least:

* jpeg2000
* ffv1
* rawvideo
* tiff

/Tomas