[FFmpeg-trac] #7736(ffmpeg:new): Incorrect video start time when muxing with opus audio
FFmpeg
trac at avcodec.org
Fri Feb 15 09:56:47 EET 2019
#7736: Incorrect video start time when muxing with opus audio
-----------------------------------+----------------------------------
Reporter: jstrot123 | Type: defect
Status: new | Priority: normal
Component: ffmpeg | Version: 4.1
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-----------------------------------+----------------------------------
== Problem description ==
In its simplest form, my goal is to create an mkv by muxing a video (vp9)
track and an audio (opus) track. Should be a simple task but I'm facing
issues with the resulting video start time.
Due to the inherent 6.5 ms (312 * 48K) delay of the opus codec, the audio
start time is expected to be -0.007ms but the video should still start at
0ms yet ffmpeg muxes with the video starting at +0.007ms:
{{{
$ ffmpeg -i track0.vp9.ivf -i track1.opus.ogg -codec copy output.mkv
ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 8 (Debian 8.2.0-10)
configuration: --disable-decoder=amrnb --disable-decoder=libopenjpeg
--disable-mips32r2 --disable-mips32r6 --disable-mips64r6 --disable-mipsdsp
--disable-mipsdspr2 --disable-mipsfpu --disable-msa --disable-libopencv
--disable-podpages --disable-sndio --disable-stripping --enable-libaom
--enable-avfilter --enable-avresample --enable-gcrypt --enable-gnutls
--enable-gpl --enable-libass --enable-libbluray --enable-libbs2b --enable-
libcaca --enable-libcdio --enable-libcodec2 --enable-libfdk-aac --enable-
libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme
--enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmp3lame
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264
--enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-
libpulse --enable-librubberband --enable-libshine --enable-libsnappy
--enable-libsoxr --enable-libspeex --enable-libtesseract --enable-
libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis
--enable-libvpx --enable-libx265 --enable-libxvid --enable-libzvbi
--enable-nonfree --enable-opencl --enable-opengl --enable-postproc
--enable-pthreads --enable-shared --enable-version3 --enable-libwebp
--incdir=/usr/include/x86_64-linux-gnu --libdir=/usr/lib/x86_64-linux-gnu
--prefix=/usr --toolchain=hardened --enable-frei0r --enable-chromaprint
--enable-libx264 --enable-libiec61883 --enable-libdc1394 --enable-vaapi
--enable-libmfx --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Input #0, ivf, from 'track0.vp9.ivf':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv),
720x480, 23.98 tbr, 23.98 tbn, 23.98 tbc
[ogg @ 0x56377efc2500] 654 bytes of comment header remain
Input #1, ogg, from 'track1.opus.ogg':
Duration: 00:10:00.49, start: 0.000000, bitrate: 409 kb/s
Stream #1:0: Audio: opus, 48000 Hz, 5.1, fltp
Metadata:
ENCODER : opusenc from opus-tools 0.1.10
ENCODER_OPTIONS : --vbr --bitrate 448
Output #0, matroska, to 'output.mkv':
Metadata:
encoder : Lavf58.20.100
Stream #0:0: Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv),
720x480, q=2-31, 23.98 tbr, 1k tbn, 23.98 tbc
Stream #0:1: Audio: opus ([255][255][255][255] / 0xFFFFFFFF), 48000
Hz, 5.1, fltp
Metadata:
ENCODER : opusenc from opus-tools 0.1.10
ENCODER_OPTIONS : --vbr --bitrate 448
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #1:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=14397 fps=0.0 q=-1.0 Lsize= 95680kB time=00:10:00.47
bitrate=1305.3kbits/s speed=1.48e+03x
video:65466kB audio:29900kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: 0.329659%
$ ffprobe -i output.mkv -show_streams 2>/dev/null | grep start_time
start_time=0.007000
start_time=-0.007000
$ ffprobe -i output.mkv -show_packets -of compact 2>/dev/null | head -2
packet|codec_type=audio|stream_index=1|pts=-7|pts_time=-0.007000|dts=-7|dts_time=-0.007000|duration=20|duration_time=0.020000|convergence_duration=N/A|convergence_duration_time=N/A|size=1737|pos=924|flags=K_
packet|codec_type=video|stream_index=0|pts=7|pts_time=0.007000|dts=7|dts_time=0.007000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=10995|pos=2668|flags=K_
}}}
If instead I use an ac3 audio track (w/o delay) then all tracks are muxed
withno delay, as expected:
{{{
$ ffmpeg -i track0.vp9.ivf -i track1.ac3 -codec copy output.mkv
...
Input #0, ivf, from 'track0.vp9.ivf':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv),
720x480, 23.98 tbr, 23.98 tbn, 23.98 tbc
[ac3 @ 0x55fdae1a1500] Estimating duration from bitrate, this may be
inaccurate
Input #1, ac3, from 'track1.ac3':
Duration: 00:10:00.48, start: 0.000000, bitrate: 448 kb/s
Stream #1:0: Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s
...
$ ffprobe -i output.mkv -show_streams 2>/dev/null | grep start_time
start_time=0.000000
start_time=0.000000
$ ffprobe -i output.mkv -show_packets -of compact 2>/dev/null | head -2
packet|codec_type=video|stream_index=0|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=10995|pos=753|flags=K_
packet|codec_type=audio|stream_index=1|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=32|duration_time=0.032000|convergence_duration=N/A|convergence_duration_time=N/A|size=1792|pos=11755|flags=K_
}}}
== Workaround ==
I tried all sorts of combinations of -itsoffset, -copyts, -start_at_zero,
-avoid_negative_ts, ... the only workaround I found (by luck) was to
provide a large (-10) negative itsoffset for the audio track and suddenly
ffmpeg stops applying a delay to the video track.
{{{
$ ffmpeg -i track0.vp9.ivf -itsoffset -10 -i track1.opus.ogg -codec copy
output.mkv
...
$ ffprobe -i output.mkv -show_streams 2>/dev/null | grep start_time
start_time=0.000000
start_time=-0.007000
$ ffprobe -i output.mkv -show_packets -of compact 2>/dev/null | head -2
packet|codec_type=video|stream_index=0|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=10995|pos=924|flags=K_
packet|codec_type=audio|stream_index=1|pts=-7|pts_time=-0.007000|dts=-7|dts_time=-0.007000|duration=20|duration_time=0.020000|convergence_duration=N/A|convergence_duration_time=N/A|size=1737|pos=11926|flags=K_
}}}
== Other observations ==
* With multiple audio tracks the -itsoffset has to be specified for each
audio track.
* Compared number of video and audio packets in ffprobe -show_packets
output and all are accounted for.
* With other itsoffset values there are weird behaviors too! With 0 to -5
it just adds a delay to the video track, with -6 to -9 it aligns both to
-7ms:
{{{
with -itsoffset 0:
start_time=0.007000
start_time=-0.007000
with -itsoffset -1:
start_time=1.007000
start_time=-0.007000
with -itsoffset -2:
start_time=2.007000
start_time=-0.007000
...
with -itsoffset -5:
start_time=5.007000
start_time=-0.007000
with -itsoffset -6 to -9:
start_time=-0.007000
start_time=-0.007000
with -itsoffset -10 to -99:59:59.99999999:
start_time=0.000000
start_time=-0.007000
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/7736>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list