[FFmpeg-devel] [RFC] Yet another subtitles open question

Josh Allmann joshua.allmann at gmail.com
Thu Oct 3 01:48:30 CEST 2013

On 2 October 2013 13:29, Clément Bœsch <u at pkh.me> wrote:
> On Wed, Oct 02, 2013 at 12:10:38PM -0700, Josh Allmann wrote:
>> On 1 October 2013 11:14, Clément Bœsch <u at pkh.me> wrote:
>> > Hi folks,
>> >
>> > I have definitely better to do, and I still hate subtitles, but we need to
>> > make some changes, and I'm willing to help.
>> >
>> > Shell
>> > -----
>> >
>> > So here it is, I'd like to move on with the API. Currently we use
>> > AVSubtitle (and its AVSubtitleRects) which is defined in lavc. We wanted
>> > at some point to move it to libavutil, eventually with slight changes.
>> > OTOH, the idea of using the AVFrame was disregarded quickly for various
>> > reasons. The thing is, it might have various benefits:
>> >
>> >  - since it doesn't introduce a new struct and its own semantic, it will
>> >    ease a lot its integration with for instance libavfilter, and probably
>> >    allow various code refactoring
>> >
>> >  - AVFrames have a ref counting system: if audio and video are ref
>> >    counted, we expect the subtitles to be as well, otherwise it can become
>> >    a source of pain for users
>> >
>> >  - using AVFrame also suggest the idea that we could embed subtitles data
>> >    within existing AVFrame: think closed captioning. "Freelance" subtitles
>> >    OTOH would just use a empty/zero frame shell. Note that this conflicts
>> >    with the ref counting idea since it can't share the data buffers.
>> >
>> > Opinion on this?
>> >
>> In Libav we're thinking of introducing an opaque field into AVFrame
>> for other purposes, and I'm thinking of leveraging it for
>> subtitle-related things there.
> Can you detail a bit more? I'm not sure I follow you with the opaque
> field. It sounds like it will require the AVSubtitle to be moved to
> libavutil anyway.

More or less, yes. There is concern about adding extra single-purpose
fields to AVFrame that are only used for subtitles, when the API user
could just as easily access an AVSubtitle (or similar). So this way,
we maintain the API/external semantics of AVFrame while keeping the
internal organization of subtitles distinct from AVFrame. The opaque
field could be re-purposed for other types of streams as well.

This idea is just preliminary, we need to consider more how that would
interact with ref counting, etc.


More information about the ffmpeg-devel mailing list