[FFmpeg-devel] [PATCH 12/18] avformat/hls: parse ID3 timestamps for elementary audio streams
anssi.hannula at iki.fi
Tue Dec 31 02:04:33 CET 2013
30.12.2013 16:12, Michael Niedermayer kirjoitti:
> On Mon, Dec 30, 2013 at 01:14:26PM +0200, Anssi Hannula wrote:
>> HLS provides MPEG TS timestamps via ID3 tags in the beginning of each
>> segment of elementary audio streams.
>> Signed-off-by: Anssi Hannula <anssi.hannula at iki.fi>
>> This is a bit hacky, but I could not see any better way.
> this seems to break some hls streams
> for example:
> didnt investigate why
The ID3 tag in the beginning of each segment seems to contain an
attached picture (APIC) in addition to the timestamp, so the ID3 tag is
too large for the id3 hack to fully intercept (since the hack currently
depends on the ID3 tag fitting in the caller-provided buffer), so the
caller gets 0 bytes.
The APIC is currently parsed to a static mjpeg stream by
ff_id3v2_parse_apic() in avformat_open_input(). I guess it could
theoretically change mid-stream as well, though it might not currently
happen in practice (no idea, though (13) suggests it does not).
Before the patch it works because avformat_open_input() reads metadata
in the beginning of input. It of course will not detect the mid-stream
metadata changes (in the beginning of segments), if there are such streams.
I guess there are several alternative ways to fix this:
a) Make the hack properly request additional bytes into its own buffer
so that the entire ID3 tag can be read, and get the picture shipped
over to the attached picture stream (and maybe any changed text
metadata as well?). Not sure how that should work with changing
pictures, though, as AV_DISPOSITION_ATTACHED_PIC says "single
packet". Or alternatively just not create it is an attachment in the
first place, but as a regular video stream. Well, I guess for now
I could just ignore all data after the first ID3 tag, and deal
with the changing stuff if it appears...
b) Assume the timestamp we want is in the beginning of the packet (so
we do not need to request any additional bytes) and then pass the
full ID3 tag along after extracting the timestamp we wanted. However,
I guess that is a pretty big assumption, and of course this way we
miss any metadata in mid-stream ID3 tags (though that is a minor
problem as such streams may not exist).
c) Some entirely different solution to this ID3 timestamp thing
altogether, preferably something that would also allow us to drop
the manual recalculation of timestamps in this patch (by leveraging
the lavf timestamp regeneration code).
I already dabbled a bit with a private HLS ID3 tagged audio demuxer
but the issue was that from the perspective of such a demuxer, the
position of the ID3 tags is only known by the AVIOContext, and I
could see no easy way to communicate the position of ID3 tags to
the demuxer externally (well, I guess I could rather easily just
access the private demuxer priv_data.. but that would again be
quite hacky and probably would make no sense to have the HLS ID3
audio demuxer even registered publicly in allformats.c..)
Hmm, continuing on the subdemuxer idea, maybe a HLS ID3 audio demuxer
could even handle the segment downloading itself so it would then
know where to expect ID3 tags, sharing code with the main HLS demuxer
but without any extra communication between them.
I don't see how the probing would work, exactly, though... i.e.
how would the HLS ID3 subdemuxer be selected? Just probing for the
ID3 timestamp tag in probe() would not be enough, since at that point
the main HLS demuxer is already handling segments via custom
AVIOContext, so it would have to handover somehow, which is again
in the hack category... damn, I thought I had it this time.
I guess I'm going to look at implementing (a), unless you can devise a
workable idea along the lines of (c)...
More information about the ffmpeg-devel