[FFmpeg-trac] #6915(avformat:open): DASH audio segments duration doesn't match exactly with video segments duration.
FFmpeg
trac at avcodec.org
Thu Dec 21 18:17:54 EET 2017
#6915: DASH audio segments duration doesn't match exactly with video segments
duration.
------------------------------------+-------------------------------------
Reporter: beloko | Owner: stevenliu
Type: defect | Status: open
Priority: normal | Component: avformat
Version: git-master | Resolution:
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
------------------------------------+-------------------------------------
Comment (by stevenliu):
Replying to [comment:24 j_karthic]:
> Replying to [comment:22 stevenliu]:
> > Replying to [comment:21 j_karthic]:
> > > Replying to [comment:19 stevenliu]:
> > > > Replying to [comment:18 stevenliu]:
> > > > > Replying to [comment:17 j_karthic]:
> > > > > > Replying to [comment:16 stevenliu]:
> > > > > > > Replying to [comment:15 j_karthic]:
> > > > > > > > Replying to [comment:14 stevenliu]:
> > > > > > > > > Replying to [comment:13 j_karthic]:
> > > > > > > > > > Replying to [comment:12 stevenliu]:
> > > > > > > > > > > Replying to [comment:11 j_karthic]:
> > > > > > > > > > > > Replying to [comment:10 stevenliu]:
> > > > > > > > > > > > > Replying to [comment:9 j_karthic]:
> > > > > > > > > > > > > > Replying to [comment:8 stevenliu]:
> > > > > > > > > > > > > > > Replying to [comment:7 j_karthic]:
> > > > > > > > > > > > > > > > Well, if we round the target duration to
'''nearest integer''' as per the spec, part of the problem is resolved.
Right now we are '''ceiling to the upper integer''' which is not according
to HLS specifications.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > {{{
> > > > > > > > > > > > > > > > 4.3.3.1. EXT-X-TARGETDURATION
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The EXT-X-TARGETDURATION tag specifies
the maximum Media Segment
> > > > > > > > > > > > > > > > duration. The EXTINF duration of each
Media Segment in the Playlist
> > > > > > > > > > > > > > > > file, when rounded to the nearest
integer, MUST be less than or equal
> > > > > > > > > > > > > > > > to the target duration; longer segments
can trigger playback stalls
> > > > > > > > > > > > > > > > or other errors. It applies to the
entire Playlist file. Its format
> > > > > > > > > > > > > > > > is:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > #EXT-X-TARGETDURATION:<s>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > where s is a decimal-integer indicating
the target duration in
> > > > > > > > > > > > > > > > seconds. The EXT-X-TARGETDURATION tag
is REQUIRED.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > }}}
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > So as per the spec, the
EXT-X-TARGETDURATION should rounded to 4 instead of 5.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > But when I submitted a patch for fixing
such an issue in hlsenc, it was rejected by Steven. See thread
http://ffmpeg.org/pipermail/ffmpeg-devel/2017-September/215630.html more
details. During that time, I was not able to provide a concrete example
with mediastreamvalidator like @beloko has done now. But I thought the
spec was very clear about it without any room for confusion.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > @stevenliu
> > > > > > > > > > > > > > > > If you have a change of mind after seeing
these results, please let me know. Maybe I will send a new patch which
fixes the target duration in dashenc, and we can take forward our
discussions there.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The spec said "EXT-X-TARGETDURATION must >
EXTINF", we have talk and you have sent twice patch about it. you can see
the spec said "MUST be less than or equal to the target duration.",
attention the MUST, MUST, MUST, we should don't care the tool, we should
care the spec. If the tool different with the spec, that maybe apple's
mistake, you should push apple.
> > > > > > > > > > > > > > I think you missing the important line
"'''when rounded to the nearest integer'''" in the spec. I am also only
talking about the spec. This is just a pure argument, about interpreting
that English line. This is where I feel some other people in the ffmpeg
community should also pitch in moderate a discussion, rather than leaving
it as "'''Maintainer is always right'''".
> > > > > > > > > > > > > '''Maintainer''' not always right, but can you
give me the mean '''MUST be less than or equal to the target duration'''
how should i understand it?
> > > > > > > > > > > > "'''when rounded to the nearest integer, MUST be
less than or equal to the target duration'''" is in one sentence. That
should be read in conjunction with the phrase "when rounded to the nearest
integer". It means after rounding duration to the nearest integer, it MUST
be less than or equal to the target duration.
> > > > > > > > > > > how should i understand the '''target duration'''?
> > > > > > > > > > > do you mean, when the '''EXTINF''' is
'''1.080000''', the '''EXT-X-TARGETDURATION''' should equal to '''1''' ?
> > > > > > > > > > As per the spec, Yes.
> > > > > > > > > I coding for hls from VERSION 1, from old VERSION,
https://tools.ietf.org/html/draft-pantos-http-live-streaming-03
> > > > > > > > > about the EXT-X-TARGETDURATION said:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > {{{
> > > > > > > > > 3.2.1. EXT-X-TARGETDURATION
> > > > > > > > >
> > > > > > > > > The EXT-X-TARGETDURATION tag specifies the maximum
media file
> > > > > > > > > duration. The EXTINF duration of each media file in
the Playlist
> > > > > > > > > file MUST be less than or equal to the target
duration. This tag
> > > > > > > > > MUST appear once in the Playlist file. Its format
is:
> > > > > > > > >
> > > > > > > > > #EXT-X-TARGETDURATION:<s>
> > > > > > > > >
> > > > > > > > > where s is an integer indicating the target duration
in seconds.
> > > > > > > > > }}}
> > > > > > > > >
> > > > > > > > > so, i always understand it to EXT-X-TARGETDURATION must
large than EXTINF,
> > > > > > > > > I have sent a Email to hls team of apple for check this
description, maybe there have some misunderstanding words here.
> > > > > > > > In that version of HLS, floating point segment duration
was not supported. The segment durations were already integers. Hence
there is no question of rounding, when target duration was being defined.
> > > > > > > > But later when the floating point segment duration was
supported in HLS, the definition for target duration was modified to
specify "when rounded to nearest integer" to handle floating point segment
durations.
> > > > > > >
> > > > > > > Sorry for my poor English, but i think the '''when rounded
to the nearest integer''' is description for '''The EXTINF duration of
each Media Segment in the Playlist file,''', because the they in one
sentence, and the next sentence said '''longer segments can trigger
playback stalls or other errors''', so, the reason to make there have no
'''longer segments can trigger playback stalls or other errors''' error,
not use lrint.
> > > > > > > When EXTINF is 1.080000, set the EXT-X-TARGETDURATION to 2.
> > > > > > Well, let me explain that entire sentence as a pseudo code.
> > > > > >
> > > > > > {{{
> > > > > > The EXTINF duration of each Media Segment in the Playlist
> > > > > > file, when rounded to the nearest integer, MUST be less than
or equal
> > > > > > to the target duration; longer segments can trigger playback
stalls
> > > > > > or other errors.
> > > > > > }}}
> > > > > >
> > > > > > {{{
> > > > > > for each media segment {
> > > > > > if (round(EXTINF_duration) <= target_duration) { //when
rounded to the nearest integer, MUST be less than or equal to the target
duration
> > > > > > Everything is fine here
> > > > > > } else { // longer segments ...
> > > > > > //Basically when (round(EXTINF_duration) > target_duration)
> > > > > > playback stalls or other errors can be triggered;
> > > > > > }
> > > > > > }
> > > > > > }}}
> > > > > >
> > > > > > '''longer segments can trigger playback stalls or other
errors''', is still part of the same sentence. This means longer after
rounding to the nearest integer.
> > > > >
> > > > > what will happen when just round the target duration ？i think
the EXTINF will large than target duration
> > > > round up it is no problem, but round down will have different
result for users understand,This is why implement an api to process it for
round up.
> > > Well as long as it follows the spec, we need not worry. If the
players were implemented as per the HLS spec then they should handle this
case correctly.
> >
> > I will try to fix this problem
> Oh! I am glad that we are on the same page. Thanks for your
understanding.
> Regarding the fix, I think fixing the target duration is good enough.
For that I have already sent a patch. http://ffmpeg.org/pipermail/ffmpeg-
devel/2017-December/222745.html
> It is theoretically not possible to get audio duration to exactly
4.00000 seconds. Because an AAC frame size is 1024 which is not a divisor
of 4*48000 or 4*44100. So the audio segment can't be cut at exact 4.00000
seconds. Even the HLS spec is fine with that. In section 6.2.4. Providing
Variant Streams, it mentions that the target duration of all variants to
be same. No explicit mention for the EXTINF duration indirectly means that
those durations need not be exactly same(as it is not possible
theoretically).
>
> {{{
> Each Media Playlist in each Variant Stream MUST have the same
> target duration. The only exceptions are SUBTITLES Renditions and
> Media Playlists containing an EXT-X-I-FRAMES-ONLY tag, which MAY
> have different target durations if they have an EXT-X-PLAYLIST-
> TYPE of VOD.
> }}}
{{{
MacBook:xxx StevenLiu$ ./ffmpeg -hide_banner -i
~/bbb_sunflower_1080p_30fps_normal.mp4 -g 150 -r 100 -x264opts
"scenecut=-1" -f dash -min_seg_duration 1000000 -window_size 99999 -t 5
-hls_playlist 1 output_Steven.mpd
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from
'/Users/StevenLiu/bbb_sunflower_1080p_30fps_normal.mp4':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isomavc1
creation_time : 2013-12-16T17:44:39.000000Z
title : Big Buck Bunny, Sunflower version
artist : Blender Foundation 2008, Janus Bager Kristensen 2013
comment : Creative Commons Attribution 3.0 -
http://bbb3d.renderfarming.net
genre : Animation
composer : Sacha Goedegebure
Duration: 00:10:34.53, start: 0.000000, bitrate: 3481 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p,
1920x1080 [SAR 1:1 DAR 16:9], 2998 kb/s, 30 fps, 30 tbr, 30k tbn, 60 tbc
(default)
Metadata:
creation_time : 2013-12-16T17:44:39.000000Z
handler_name : GPAC ISO Video Handler
Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo,
s16p, 160 kb/s (default)
Metadata:
creation_time : 2013-12-16T17:44:42.000000Z
handler_name : GPAC ISO Audio Handler
Stream #0:2(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side),
fltp, 320 kb/s (default)
Metadata:
creation_time : 2013-12-16T17:44:42.000000Z
handler_name : GPAC ISO Audio Handler
Side data:
audio service type: main
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:2 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x7f7fa5085200] using SAR=1/1
[libx264 @ 0x7f7fa5085200] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[libx264 @ 0x7f7fa5085200] profile High, level 5.1
[libx264 @ 0x7f7fa5085200] 264 - core 133 r2334M a3ac64b - H.264/MPEG-4
AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html -
options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7
psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1
8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6
lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=150 keyint_min=15
scenecut=0 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0
qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[aac @ 0x7f7fa5086a00] Using a PCE to encode channel layout
[dash @ 0x7f7fa6087400] No bit rate set for stream 0
[dash @ 0x7f7fa6087400] Opening 'init-stream0.m4s' for writing
[dash @ 0x7f7fa6087400] Opening 'init-stream1.m4s' for writing
[dash @ 0x7f7fa6087400] Opening 'output_Steven.mpd.tmp' for writing
Bandwidth info not available, set audio and video bitrates
Output #0, dash, to 'output_Steven.mpd':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isomavc1
composer : Sacha Goedegebure
title : Big Buck Bunny, Sunflower version
artist : Blender Foundation 2008, Janus Bager Kristensen 2013
comment : Creative Commons Attribution 3.0 -
http://bbb3d.renderfarming.net
genre : Animation
encoder : Lavf58.3.100
Stream #0:0(und): Video: h264 (libx264), yuv420p(progressive),
1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 100 fps, 12800 tbn, 100 tbc
(default)
Metadata:
creation_time : 2013-12-16T17:44:39.000000Z
handler_name : GPAC ISO Video Handler
encoder : Lavc58.8.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #0:1(und): Audio: aac (LC), 48000 Hz, 5.1(side), fltp, 394 kb/s
(default)
Metadata:
creation_time : 2013-12-16T17:44:42.000000Z
handler_name : GPAC ISO Audio Handler
encoder : Lavc58.8.100 aac
Side data:
audio service type: main
[dash @ 0x7f7fa6087400] Opening 'chunk-stream0-00001.m4s.tmp' for
writingdrop=0 speed=0.532x
[dash @ 0x7f7fa6087400] Opening 'chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'output_Steven.mpd.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'chunk-stream0-00002.m4s.tmp' for
writingdrop=0 speed=0.526x
[dash @ 0x7f7fa6087400] Opening 'chunk-stream1-00002.m4s.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'output_Steven.mpd.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'chunk-stream0-00003.m4s.tmp' for
writingdrop=0 speed=0.455x
[dash @ 0x7f7fa6087400] Opening 'chunk-stream1-00003.m4s.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'output_Steven.mpd.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'chunk-stream0-00004.m4s.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'chunk-stream1-00004.m4s.tmp' for writing
[dash @ 0x7f7fa6087400] Opening 'output_Steven.mpd.tmp' for writing
frame= 500 fps= 41 q=-1.0 Lsize=N/A time=00:00:05.01 bitrate=N/A dup=354
drop=0 speed=0.41x
video:365kB audio:241kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: unknown
[libx264 @ 0x7f7fa5085200] frame I:4 Avg QP:16.78 size: 20153
[libx264 @ 0x7f7fa5085200] frame P:150 Avg QP:20.52 size: 1625
[libx264 @ 0x7f7fa5085200] frame B:346 Avg QP:18.64 size: 141
[libx264 @ 0x7f7fa5085200] consecutive B-frames: 5.4% 5.2% 5.4% 84.0%
[libx264 @ 0x7f7fa5085200] mb I I16..4: 92.8% 3.1% 4.1%
[libx264 @ 0x7f7fa5085200] mb P I16..4: 6.0% 1.9% 0.1% P16..4: 2.5%
0.2% 0.2% 0.0% 0.0% skip:89.2%
[libx264 @ 0x7f7fa5085200] mb B I16..4: 0.2% 0.0% 0.0% B16..8: 0.6%
0.0% 0.0% direct: 0.2% skip:99.0% L0:33.2% L1:65.7% BI: 1.1%
[libx264 @ 0x7f7fa5085200] 8x8 transform intra:17.9% inter:71.0%
[libx264 @ 0x7f7fa5085200] coded y,uvDC,uvAC intra: 3.2% 11.7% 2.2% inter:
0.2% 0.7% 0.1%
[libx264 @ 0x7f7fa5085200] i16 v,h,dc,p: 83% 12% 3% 1%
[libx264 @ 0x7f7fa5085200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 11% 62% 1%
2% 1% 2% 1% 1%
[libx264 @ 0x7f7fa5085200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 26% 22% 28% 4%
4% 4% 4% 3% 4%
[libx264 @ 0x7f7fa5085200] i8c dc,h,v,p: 77% 14% 9% 0%
[libx264 @ 0x7f7fa5085200] Weighted P-Frames: Y:22.7% UV:22.0%
[libx264 @ 0x7f7fa5085200] ref P L0: 83.9% 4.5% 9.7% 1.5% 0.4%
[libx264 @ 0x7f7fa5085200] ref B L0: 73.8% 24.9% 1.3%
[libx264 @ 0x7f7fa5085200] ref B L1: 97.2% 2.8%
[libx264 @ 0x7f7fa5085200] kb/s:597.11
[aac @ 0x7f7fa5086a00] Qavg: 190.813
MacBook:xxx StevenLiu$ mediastreamvalidator master.m3u8
mediastreamvalidator: Version 1.2(170822)
[master.m3u8] Started loading root playlist
[media_1.m3u8] Started loading media playlist
Can't deal with multiple sample timings per sample buffer
[media_1.m3u8] All media files delivered and have end tag, stopping
--------------------------------------------------------------------------------
media_1.m3u8
--------------------------------------------------------------------------------
Processed 4 out of 4 segments
Average segment duration: 1.250000
Total segment bitrates (all discontinuities): average: 397.05 kb/s, max:
405.60 kb/s
Playlist max bitrate: 394.000000 kb/s
Audio Group ID: AUDIO
Discontinuity: sequence: 0, parsed segment count: 4 of 4, duration: 5.000
sec, average: 397.05 kb/s, max: 405.60 kb/s
Track ID: 1
Audio Codec: AAC-LC
Audio sample rate: 48000 Hz
Audio channels: 0
Audio channel layout: (null)
--------------------------------------------------------------------------------
MUST fix issues
--------------------------------------------------------------------------------
Error: Playlist vs segment duration mismatch
--> Detail: Segment duration 2.0053, Playlist duration: 1.4933
--> Source: media_1.m3u8 - chunk-stream1-00003.m4s:73981 at 0
MacBook:xxx StevenLiu$ git diff
diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c
index 5687530f2d..5368a2334c 100644
--- a/libavformat/dashenc.c
+++ b/libavformat/dashenc.c
@@ -358,7 +358,7 @@ static void output_segment_list(OutputStream *os,
AVIOContext *out, DASHContext
Segment *seg = os->segments[i];
double duration = (double) seg->duration / timescale;
if (target_duration <= duration)
- target_duration = hls_get_int_from_double(duration);
+ target_duration = lrint(duration);
}
ff_hls_write_playlist_header(out_hls, 6, -1, target_duration,
MacBook:xxx StevenLiu$
}}}
I think the problem is not the EXTINF and the EXT-X-TARGETDURATION
problem, the problem the playlist's target duration not same.
>
> > > >
> > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
--
Ticket URL: <https://trac.ffmpeg.org/ticket/6915#comment:26>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list