[FFmpeg-devel] [PATCH] mov.c: allow reading fragment start dts/pts from fragmented mp4
Mika Raento
mikie at iki.fi
Sat Oct 11 19:38:14 CEST 2014
On 11 October 2014 20:20, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Sat, Oct 11, 2014 at 06:43:48PM +0300, Mika Raento wrote:
>> This introduces a new option to the mov demuxer: -use_mfra_for
>> (pts|dts). When it's given and moofs and a MFRA are present, the MFRA's
>> TFRAs are read for fragment start times.
>>
>> Unfortunately some programs that produce fragmented mp4s use the TFRA
>> time field for dts and some for pts. There is no realistic way to detect
>> which is the case, hence the responsibility is punted onto the user.
>> This also means that no behavioural change is enabled by default - you
>> must pass either dts or pts for anything to happen.
>>
>> Without this change, timestamps for some discontinuous fragmented mp4 are
>> wrong, and cause audio/video desync and are not usable for generating
>> HLS.
>> ---
> [...]
>> @@ -3943,6 +4111,15 @@ static const AVOption options[] = {
>> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>> {"ignore_editlist", "", offsetof(MOVContext, ignore_editlist), FF_OPT_TYPE_INT, {.i64 = 0},
>> 0, 1, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM},
>> + {"use_mfra_for",
>> + "use mfra for fragment timestamps",
>> + offsetof(MOVContext, use_mfra_for), FF_OPT_TYPE_INT, {.i64 = 0},
>> + 0, FF_MOV_FLAG_MFRA_PTS, AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_DECODING_PARAM,
>> + "use_mfra_for"},
>
> maybe a FF_MOV_FLAG_MFRA_AUTO which is set by default could be used to
> guess based on some encoder/muxer identification what to do.
>
> do the files which need this option have something that idenifies
> them? like in metadata, compatible brands or other?
> and can you share a file/testcase which needs this patch ?
I can share a file privately (copyrighted). I do not have a reduced
test case. Let me know if you think that's helpful.
The files come from a commercial packager that is getting
discontinuous input from a commercial transcoder. My understanding is
that, the timestamp issue comes from the packager, rather than the
transcoder. The underlying software in the packager is Code-shop's
Unified Streaming, but it's been fed from proprietary software.
This is what ffprobe says, it's not very specific:
(multiprofile-streamers)mikie at universe:~/devel/multiprofile-streamers$
ffprobe -i ../multiprofile-fixer/hls-disco/50603088.ismv
ffprobe version N-66765-g67d95df Copyright (c) 2007-2014 the FFmpeg developers
built on Oct 11 2014 18:42:41 with Apple LLVM version 6.0
(clang-600.0.51) (based on LLVM 3.5svn)
configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD
--enable-shared --enable-pthreads --enable-gpl --enable-version3
--enable-nonfree --enable-hardcoded-tables --enable-avresample
--enable-vda --cc=clang --host-cflags= --host-ldflags=
--enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid
--enable-debug --disable-stripping --enable-libass
libavutil 54. 10.100 / 54. 10.100
libavcodec 56. 4.101 / 56. 4.101
libavformat 56. 9.100 / 56. 9.100
libavdevice 56. 1.100 / 56. 1.100
libavfilter 5. 1.103 / 5. 1.103
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.100 / 3. 1.100
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 1.100 / 53. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from
'../multiprofile-fixer/hls-disco/50603088.ismv':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isomiso2
Duration: 02:17:00.14, start: 0.000000, bitrate: 4411 kb/s
Stream #0:0(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
stereo, fltp, 124 kb/s (default)
Metadata:
handler_name : Sound Handler
Stream #0:1(mul): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
stereo, fltp, 29 kb/s (default)
Metadata:
handler_name : Sound Handler
Stream #0:2(und): Video: h264 (Main) (avc1 / 0x31637661),
yuv420p(tv, bt470bg), 480x270 [SAR 1:1 DAR 16:9], 597 kb/s, 25 fps, 25
tbr, 10000k tbn, 50 tbc (default)
Metadata:
handler_name : Video Handler
encoder : AVC Coding
Stream #0:3(und): Video: h264 (Main) (avc1 / 0x31637661),
yuv420p(tv, bt470bg), 640x360 [SAR 1:1 DAR 16:9], 1195 kb/s, 25 fps,
25 tbr, 10000k tbn, 50 tbc (default)
Metadata:
handler_name : Video Handler
encoder : AVC Coding
Stream #0:4(und): Video: h264 (High) (avc1 / 0x31637661),
yuv420p(tv, bt470bg), 768x432 [SAR 1:1 DAR 16:9], 2487 kb/s, 25 fps,
25 tbr, 10000k tbn, 50 tbc (default)
Metadata:
handler_name : Video Handler
encoder : AVC Coding
>
> Also ISO/IEC 14496-12:2008(E) seems to say this:
> This box provides only a hint as to where random access points are; the movie fragments themselves are
> definitive. It is recommended that readers take care in both locating and using this box as modifications to the
> file after it was created may render either the pointers, or the declaration of random access points, incorrect.
>
> so if i understand it correctly, using these boxes requires some care
> like maybe checking for some brand or muxer/encoder which makes further
> gurantees about their validity.
>From the differences between the packagers and the different
interpretations of the spec, I'm not very eager to attempt
autodetection.
Consider ffmpeg: we'll at some point change the fragmentation code to
use pts. Whether that'll be visible in the file metadata is unclear.
I'm not sure the MFRA is meant to be used to fix up the timestamps, it
just that for these files it's the only source for them.
I'll ask our vendor for their opinion too, though I'm not too
optimistic about them saying anything useful.
Mika
>
> Thanks
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Rewriting code that is poorly written but fully understood is good.
> Rewriting code that one doesnt understand is a sign that one is less smart
> then the original author, trying to rewrite it will not make it better.
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
More information about the ffmpeg-devel
mailing list