[FFmpeg-trac] #8366(avformat:new): Audio ticks and video/audio out-of-sync when using MXF files with pre-charge

FFmpeg trac at avcodec.org
Mon Nov 18 13:55:55 EET 2019

#8366: Audio ticks and video/audio out-of-sync when using MXF files with pre-
             Reporter:  HenkDemper  |                    Owner:
                 Type:  defect      |                   Status:  new
             Priority:  normal      |                Component:  avformat
              Version:  git-master  |               Resolution:
             Keywords:  MXF         |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |

Comment (by Tjoppen):

 Replying to [comment:6 HenkDemper]:
 > Hello Tjoppen,
 > I have & read the MXF Book, know & implemented ST377, but I think the
 original design idea about OP1a was to have a single '''logical''' clip
 from start-to-end. Anyhow all places I can find these simple pre-charged
 files (so only including pre-charge and roll-out) they are marked as OP1a.
 There is just no other technical way to achieve this and when exchanging
 files we (broadcast industry) don't want to label them (in this case) as
 anything other than OP1a, exactly because what you state: there are not so
 much MXF readers available that (claim) to support anything beyond OP1a...

 This may well be the intent, but it's also not what the spec says :) But,
 !OperationalPattern is just informative, the truth for any particular file
 is in how its MPs and FPs are laid out.

 > If we would like to be 100% sure, I might reach out to 'Mr MXF' (Bruce
 Devlin) or others on the original MXF design team (that came up with the
 OP's) and ask him/them for his/their advise on the subject... but given
 that these files are actually frequently used/seen in practice it won't
 change the use case, so maybe wasted time... In the end the goal should be
 to let FFmpeg (/libavformat/VLC/...) cope with as many files as possible.
 Agree ?

 FFmpeg can certainly be made to support at least this class of files. At
 some point it's better to let things like libmelt take over

 > > No, both audio and video have Origin > 0. It you only cut the audio
 then the output will be horribly out of sync :)
 > Well, the video Origin is already used correctly in FFmpeg: the pre-
 charge video frames (amount: Origin) are used used to produce the first
 visible frame, but never displayed itself. So it would 'only' need
 handling for the audio case, which is now out-of-sync with video if there
 is pre-charge...

 I didn't notice this when looking at the file, but perhaps ffplay does
 sync differently from ffmpeg.

 > > What if Origin is very large? I do not want the same kind of hack that
 libavformat/mov.c does for edit lists, that crap doesn't belong in a
 > We (as in our software) placed this pre-charge logic in the MXF file
 reader (because it is specific to MXF files and handled differently in for
 instance QuickTime files as mentioned). We first 'ask' the MXF reader
 which 'media frame' (Source Package, Index Table) to fetch for 'display
 frame' (viewable, Material Package) and then ask it to fetch that 'media
 frame' (for audio that is. for video we need to dig back to fill the
 decoder with pre-charge video frames). That would seem the proper 2-step
 way supporting any media (single/multi-item) in a MXF (or any other
 wrapper) file, with a split responsibility between the wrapper
 (MXF/QuickTime/...) and upper layer I would say ?
 > As mentioned, I'm not intimate with how this (timing, single/multiple
 items in editlist/timeline, media vs display frames, pre-charge, ..)
 should/is exactly be properly handled in FFmpeg, but my guess would be the
 fix for the problem on-hand would be in libavformat\mxfdec.c. Happy to
 study that code more (as a possible next step), but do not want (as you
 mention) add 'crap in the wrong places'... So open for alternative

 Well, exporting this information so that the ffmpeg CLI could make use of
 it for all streams probably wouldn't hurt. But there are some use cases to

 * remuxing MXF -> MXF
 * remuxing MXF -> MOV
 * remuxing MOV -> MXF
 * transcoding MXF, including all frames and including EDL information
 (only MOV and MXF out)
 * transcoding MXF, cutting off the start and end, no EDL (any output

 I don't think there's any other containers that support EDL stuff, but I
 may be wrong. Which of the bottom two to pick is sort of business logic..

 If support for this were added then FFmpeg could claim support for "single
 clip OP3a" perhaps

Ticket URL: <https://trac.ffmpeg.org/ticket/8366#comment:7>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

More information about the FFmpeg-trac mailing list