[FFmpeg-devel] [RFC] Format of packets for text subtitles

Nicolas George nicolas.george at normalesup.org
Thu Jun 7 22:48:14 CEST 2012

Le decadi 20 prairial, an CCXX, Clément Bœsch a écrit :
> You might want to look at
> http://gpac.wp.mines-telecom.fr/mp4box/ttxt-format-documentation/ though
> (MP4 related).

Unless I am mistaken, it does not explain the format of the packets inside
the ISOM structure. That would be the interesting point.

> About timing, the DTS is also generally set; maybe it can be useful for
> reordering subtitles?

Actually, I do not understand the point of DTS at all, but I would be
interested by explanations.

> Note: what to do about unrelated information such as comments? JACOSub for
> instance can be filled with random comments all over the file. ATM I'm
> dropping them (because it's a pain to handle). Should we "standardize"
> the way of handling this?

That could go in side_data, maybe?

> > All subtitle packets are flagged as keyframes.
> Demuxers must do it? In what case wouldn't you put them?

I guess: a demuxer for a generic format takes the flag from the container,
and we hope the file has the flag set; a demuxer for a specialized format
sets it.

> Should we make an "input subtitle encoding" option available for all
> subtitles format at a lavf top level? (available in with CONFIG_ICONV)

I am not sure whether that belongs at the top level or on a per-demuxer
basis. What format do you know, apart from MKV, MP4 and OGM, that can hold
text subtitles?

> Dropping the timing information from the packet means we might not be able
> to reconstruct it exactly based on the pts+duration, but I'm not sure
> that's really a problem.

I do not understand what you mean here.

> Also, I would actually have a CODEC_ID_SRT and a CODEC_ID_SUBRIP:
> CODEC_ID_SRT would contain the timing information + markup data, and
> CODEC_ID_SUBRIP only the markup.

Having two codec IDs to distinguish the format is an interesting idea, but
how do we tell the demuxer or whatever that we are interested in sane
CODEC_ID_SUBRIP or in compatibility CODEC_ID_SRT?

>				   Note that for instance the "SRT" format
> can have some extra coordinates data mixed with the timing of each
> event...

Could go in the side_data too.

> Yes this was a timing issue, which I indeed solved in the demuxer context
> (see 2d52ee8a1a4f9438df90f3c95a6fbfc8f6e812f3). But this kind of
> "workaround" could have been put at another level; for instance there is a
> similar issue with SAMI: the next subtitle replaces the previous one
> (there is no duration field or something), and thus you always need to
> demux two packets at a time, buffer one, etc (while we could just have put
> a duration = -1 in the packet).

A SAMI file is a dedicated format, so we can have the demuxer do more or
less what we need it to do.

It would be much more problematic if you knew a format that can contain text
subtitles interleaved with video and that relies on that trick to mark the
end of a subtitle, because we can not afford to buffer several seconds worth
of stream just to find the duration information.


  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120607/54d1d6e0/attachment.asc>

More information about the ffmpeg-devel mailing list