[FFmpeg-devel] [PATCH] avformat/hls: Added subtitle support

[wm4]

On Wed, 16 Nov 2016 11:21:59 +0000
Franklin Phillips <franklinphillips at gmx.com> wrote:

> On Wed, Nov 16, 2016 at 11:41:41AM +0100, wm4 wrote:
> > On Wed, 16 Nov 2016 10:31:10 +0000
> > Franklin Phillips <franklinphillips at gmx.com> wrote:
> >   
> > > This patch is a fix for ticket #2833.
> > > 
> > > Each subtitle segment is its own WebVTT file and has to demuxed individually.  
> >   
> > > The timing of the subtitles are not perfect but it is the best I could  
> > 
> > What does this mean?
> >   
> When I convert a mp4 file with subtitles into hls and then back into mp4, the subtitles are off by a couple 100 milliseconds usually. It seems to be caused by the first audio/video packet not having a 0 timestamp causing ffmpeg to subtract the timestamp of that first packet from all the subsequent packets.

FFmpeg determines a global start_time, which is probably used in
transcoding to offset all streams or so. But that shouldn't lead to
timestamp mismatches because it's the same offset.

Can you check the timestamps with ffprobe? Also make sure you return
subtitle packets ideally before audio/video packets with the same or a
higher timestamp. Otherwise it could happen that a subtitle packet is
too late to be rendered on video. (Though I don't know what happens
when remuxing subtitles in this scenario.)

> > > do and it also does not take into account the X-TIMESTAMP-MAP header
> > > in the WebVTT files which is needed to conform to the specification
> > > (https://tools.ietf.org/html/draft-pantos-http-live-streaming-20#section-3.5).  
> > 
> > I'm not entirely familiar how HLS works. Shouldn't this be fine as long
> > as the webvtt events have the correct "final" timestamp?
> >   
> Yes, I believe so, the fall back in the specification is to assume the value of that header is 0 meaning no offset and all the examples I've seen have that value set to 0 anyway. I don't think it's a crucial feature that needs to be done ASAP.
> 
> > > 

> > > [...]
> > > +  
> > 
> > That seems to duplicate a lot of code from the normal read path...
> >   
> I don't think there is a lot of duplication, and the duplication that there is, I don't think can be factored out cleanly due to the different ways the two methods are called and the difference in behaviour of the two methods.
> 

Well, if you say so. Still hurts a bit.