[FFmpeg-trac] #681(avfilter:open): -vf mp=pullup leads to massive A/V-desync

[FFmpeg]

#681: -vf mp=pullup leads to massive A/V-desync
-------------------------------------+-------------------------------------
             Reporter:  dericed      |                    Owner:
                 Type:  defect       |                   Status:  open
             Priority:  normal       |                Component:  avfilter
              Version:  git-master   |               Resolution:
             Keywords:  mpfilter     |               Blocked By:
  pullup                             |  Reproduced by developer:  1
             Blocking:               |
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------

Comment (by cehoyos):

 Replying to [comment:5 darkmoon]:
 > When I first found this ticket,
 [http://ffmpeg.org/trac/ffmpeg/ticket/681#comment:3 this solution] was
 your last update. When I tried it, I too experienced massive A/V desync.

 > In fact, there was desync with every other Mplayer filter I tried

 Could you elaborate?
 I don't think this is a known problem. (Or do you mean: With every other
 MPlayer inverse telecine filter except detc which unfortunately does not
 work for the real-world samples you have?)

 > ''except'' the one you quote
 [http://ffmpeg.org/trac/ffmpeg/ticket/681#comment:4 just above], and with
 the same options—
 > {{{
 > mp=detc=am=1:dr=2
 > }}}
 > for example, also causes desync.
 >
 > Section 7.2.3.4 of the Mplayer docs suggests placing the
 ''softpulldown'' filter before ''detc'' or ''ivtc'' in order to insure
 those filters have a uniformly telecine'd stream to process.

 > Additionally, I noticed that the output of ''detc'' is interlaced.

 I don't think this is correct as-such:
 Of course I believe you that you tried the filter with a sample and the
 output looked interlaced, but the point is that the filter can only work
 on clean (unedited) telecined material on which it will perform a perfect
 inverse telecine process if invoked with the correct start-pattern.

 > I was able to get good results with this filterchain:
 > {{{
 > -vf
 mp=softpulldown,mp=detc=am=0:dr=2:fr=0,yadif,setpts='N/(24000/1001*TB)' -r
 24000/1001
 > }}}

 I live in PAL-country and I am therefore certainly no expert for telecined
 material, but I believe you first have to decide if your input video is
 interlaced or telecined and then you should either use a de-interlacer
 (yadif) or try an inverse telecine process (which is currently not working
 perfectly within FFmpeg), combining them seems like a very bad idea to me.
 (Or in other words: inverse telecine is a process that recovers the
 original material without any quality loss whatsoever, while a de-
 interlacer gets broken input and tries to hide the brokenness by changing
 the video severely. Doing this on telecined input ruins every possibility
 to ever get back the original video.)

 > Ultimately, the reason for the A/V desync with the other Mplayer filters
 is the equation for ''setpts'' (='N/(24000/1001*TB)') which, according to
 [http://ffmpeg.org/ffmpeg.html#Examples-14 Section 31.2.1] of the ffmpeg
 docs, sets a fixed frame rate.

 I absolutely may miss something, but imo the output of inverse telecine
 has a fixed frame rate of 24000/1001.

 > What is needed is an equation for ''setpts'' that produces an output
 stream with the same '''''running time''''' as the input stream,
 regardless of how many frames the preceding filters drop. Perhaps someone
 with some math skills and an understanding of the ''setpts'' internal
 variables could write one for us???? :)
 >
 > (It seems likely that MEncoder includes code to make this calculation
 internally. In every discussion I have read concerning how to ivtc with
 MEncoder, there is no mention of any A/V desync problems, and no inclusion
 of any sort of a ''setpts'' filter in any of the sample command
 lines—these things are conspicuous by their absence. Instead, MEncoder has
 separate command line options for the input stream framerate, and the
 desired output framerate.)

 Note that in my experience, current MEncoder fails very badly for
 telecined material so whoever is interested in solving this problem and
 has some programming skills, should really work on porting pullup (or
 filmdint) to a native FFmpeg filter (it is imo not unlikely that
 timestamps are the only thing missing.)

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/681#comment:6>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker