[FFmpeg-devel] Internal handling of subtitles in ffmpeg

[Wolfram Gloger]

Hi,

> I have noticed that subtitles are sent internally in ffmpeg between
> decoders and encoders as bitmaps, which is a bit limiting if one were to
> introduce text-based subtitles.

Text-based subtitles _are_ already supported, please check out
CODEC_ID_TEXT et.al.

> 1. Extend the "format" member of the AVSubtitle struct to include a text
> format and eventually specific types of graphic formats

The latter would of course be possible, but if that would be
required I would suggest to use AVPicture instead as that could
be AVFilter'ed more easily.

> 3. Create a general purpose function to convert text into a bitmap

Hmm, that would require complicated font handling.

> 4. Add an interface for subtitle codecs to publish what types of
> subtitle inputs and outputs they can (or must) have
> 5. Add some logic in ffmpeg to determine how to convert the subtitle
> types between different codecs

IMHO the infrastructure for this is already there, there could well
be an encoder to take CODEC_ID_TEXT as input and CODEC_ID_DVDSUB
as output.

> My idea, basically, is that if one was to transcode a text-based
> subtitle into another text-based subtitle, ffmpeg would recognize this
> and pass the text between the codecs.

This, too, is already there; the text is passed as timestamped packets
containing UTF8 strings.

> Also, I've noticed that some of the subtitle support is incomplete.

That is indeed very true.  For example, there is still no support for
discontinuous timestamps in ffmpeg.c and some parts of libavformat.
Transcoding a stream with subtitles is therefore bound to fail, for
various reasons.  I have posted patches for this, check the archives.

> For
> VOBSUB, .idx-files containing the timestamps are not supported and
> the .ifo files for subtitle palette on the DVDs aren't used either.

Those, too, would be difficult to add because "several input files for one
input context" are not supported AFAICS.

Regards,
Wolfram.