[FFmpeg-trac] #7736(ffmpeg:new): Incorrect video start time when muxing with opus audio

FFmpeg trac at avcodec.org
Fri Feb 15 09:56:47 EET 2019


#7736: Incorrect video start time when muxing with opus audio
-----------------------------------+----------------------------------
             Reporter:  jstrot123  |                     Type:  defect
               Status:  new        |                 Priority:  normal
            Component:  ffmpeg     |                  Version:  4.1
             Keywords:             |               Blocked By:
             Blocking:             |  Reproduced by developer:  0
Analyzed by developer:  0          |
-----------------------------------+----------------------------------
 == Problem description ==

 In its simplest form, my goal is to create an mkv by muxing a video (vp9)
 track and an audio (opus) track. Should be a simple task but I'm facing
 issues with the resulting video start time.

 Due to the inherent 6.5 ms (312 * 48K) delay of the opus codec, the audio
 start time is expected to be -0.007ms but the video should still start at
 0ms yet ffmpeg muxes with the video starting at +0.007ms:

 {{{
 $ ffmpeg -i track0.vp9.ivf -i track1.opus.ogg -codec copy output.mkv
 ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
   built with gcc 8 (Debian 8.2.0-10)
   configuration: --disable-decoder=amrnb --disable-decoder=libopenjpeg
 --disable-mips32r2 --disable-mips32r6 --disable-mips64r6 --disable-mipsdsp
 --disable-mipsdspr2 --disable-mipsfpu --disable-msa --disable-libopencv
 --disable-podpages --disable-sndio --disable-stripping --enable-libaom
 --enable-avfilter --enable-avresample --enable-gcrypt --enable-gnutls
 --enable-gpl --enable-libass --enable-libbluray --enable-libbs2b --enable-
 libcaca --enable-libcdio --enable-libcodec2 --enable-libfdk-aac --enable-
 libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme
 --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmp3lame
 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264
 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-
 libpulse --enable-librubberband --enable-libshine --enable-libsnappy
 --enable-libsoxr --enable-libspeex --enable-libtesseract --enable-
 libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis
 --enable-libvpx --enable-libx265 --enable-libxvid --enable-libzvbi
 --enable-nonfree --enable-opencl --enable-opengl --enable-postproc
 --enable-pthreads --enable-shared --enable-version3 --enable-libwebp
 --incdir=/usr/include/x86_64-linux-gnu --libdir=/usr/lib/x86_64-linux-gnu
 --prefix=/usr --toolchain=hardened --enable-frei0r --enable-chromaprint
 --enable-libx264 --enable-libiec61883 --enable-libdc1394 --enable-vaapi
 --enable-libmfx --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
   libavutil      56. 22.100 / 56. 22.100
   libavcodec     58. 35.100 / 58. 35.100
   libavformat    58. 20.100 / 58. 20.100
   libavdevice    58.  5.100 / 58.  5.100
   libavfilter     7. 40.101 /  7. 40.101
   libavresample   4.  0.  0 /  4.  0.  0
   libswscale      5.  3.100 /  5.  3.100
   libswresample   3.  3.100 /  3.  3.100
   libpostproc    55.  3.100 / 55.  3.100
 Input #0, ivf, from 'track0.vp9.ivf':
   Duration: N/A, start: 0.000000, bitrate: N/A
     Stream #0:0: Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv),
 720x480, 23.98 tbr, 23.98 tbn, 23.98 tbc
 [ogg @ 0x56377efc2500] 654 bytes of comment header remain
 Input #1, ogg, from 'track1.opus.ogg':
   Duration: 00:10:00.49, start: 0.000000, bitrate: 409 kb/s
     Stream #1:0: Audio: opus, 48000 Hz, 5.1, fltp
     Metadata:
       ENCODER         : opusenc from opus-tools 0.1.10
       ENCODER_OPTIONS : --vbr --bitrate 448
 Output #0, matroska, to 'output.mkv':
   Metadata:
     encoder         : Lavf58.20.100
     Stream #0:0: Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv),
 720x480, q=2-31, 23.98 tbr, 1k tbn, 23.98 tbc
     Stream #0:1: Audio: opus ([255][255][255][255] / 0xFFFFFFFF), 48000
 Hz, 5.1, fltp
     Metadata:
       ENCODER         : opusenc from opus-tools 0.1.10
       ENCODER_OPTIONS : --vbr --bitrate 448
 Stream mapping:
   Stream #0:0 -> #0:0 (copy)
   Stream #1:0 -> #0:1 (copy)
 Press [q] to stop, [?] for help
 frame=14397 fps=0.0 q=-1.0 Lsize=   95680kB time=00:10:00.47
 bitrate=1305.3kbits/s speed=1.48e+03x
 video:65466kB audio:29900kB subtitle:0kB other streams:0kB global
 headers:0kB muxing overhead: 0.329659%

 $ ffprobe -i output.mkv -show_streams 2>/dev/null | grep start_time
 start_time=0.007000
 start_time=-0.007000

 $ ffprobe -i output.mkv -show_packets -of compact 2>/dev/null | head -2
 packet|codec_type=audio|stream_index=1|pts=-7|pts_time=-0.007000|dts=-7|dts_time=-0.007000|duration=20|duration_time=0.020000|convergence_duration=N/A|convergence_duration_time=N/A|size=1737|pos=924|flags=K_
 packet|codec_type=video|stream_index=0|pts=7|pts_time=0.007000|dts=7|dts_time=0.007000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=10995|pos=2668|flags=K_
 }}}

 If instead I use an ac3 audio track (w/o delay) then all tracks are muxed
 withno delay, as expected:

 {{{
 $ ffmpeg -i track0.vp9.ivf -i track1.ac3 -codec copy output.mkv
 ...
 Input #0, ivf, from 'track0.vp9.ivf':
   Duration: N/A, start: 0.000000, bitrate: N/A
     Stream #0:0: Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv),
 720x480, 23.98 tbr, 23.98 tbn, 23.98 tbc
 [ac3 @ 0x55fdae1a1500] Estimating duration from bitrate, this may be
 inaccurate
 Input #1, ac3, from 'track1.ac3':
   Duration: 00:10:00.48, start: 0.000000, bitrate: 448 kb/s
     Stream #1:0: Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s
 ...

 $ ffprobe -i output.mkv -show_streams 2>/dev/null | grep start_time
 start_time=0.000000
 start_time=0.000000

 $ ffprobe -i output.mkv -show_packets -of compact 2>/dev/null | head -2
 packet|codec_type=video|stream_index=0|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=10995|pos=753|flags=K_
 packet|codec_type=audio|stream_index=1|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=32|duration_time=0.032000|convergence_duration=N/A|convergence_duration_time=N/A|size=1792|pos=11755|flags=K_
 }}}

 == Workaround ==

 I tried all sorts of combinations of -itsoffset, -copyts, -start_at_zero,
 -avoid_negative_ts, ... the only workaround I found (by luck) was to
 provide a large (-10) negative itsoffset for the audio track and suddenly
 ffmpeg stops applying a delay to the video track.


 {{{
 $ ffmpeg -i track0.vp9.ivf -itsoffset -10 -i track1.opus.ogg -codec copy
 output.mkv
 ...
 $ ffprobe -i output.mkv -show_streams 2>/dev/null | grep start_time
 start_time=0.000000
 start_time=-0.007000

 $ ffprobe -i output.mkv -show_packets -of compact 2>/dev/null | head -2
 packet|codec_type=video|stream_index=0|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=10995|pos=924|flags=K_
 packet|codec_type=audio|stream_index=1|pts=-7|pts_time=-0.007000|dts=-7|dts_time=-0.007000|duration=20|duration_time=0.020000|convergence_duration=N/A|convergence_duration_time=N/A|size=1737|pos=11926|flags=K_
 }}}


 == Other observations ==

 * With multiple audio tracks the -itsoffset has to be specified for each
 audio track.

 * Compared number of video and audio packets in ffprobe -show_packets
 output and all are accounted for.

 * With other itsoffset values there are weird behaviors too! With 0 to -5
 it just adds a delay to the video track, with -6 to -9 it aligns both to
 -7ms:

 {{{
 with -itsoffset 0:
 start_time=0.007000
 start_time=-0.007000

 with -itsoffset -1:
 start_time=1.007000
 start_time=-0.007000

 with -itsoffset -2:
 start_time=2.007000
 start_time=-0.007000

 ...
 with -itsoffset -5:
 start_time=5.007000
 start_time=-0.007000

 with -itsoffset -6 to -9:
 start_time=-0.007000
 start_time=-0.007000

 with -itsoffset -10 to -99:59:59.99999999:
 start_time=0.000000
 start_time=-0.007000
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/7736>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list