[FFmpeg-devel] Internal handling of subtitles in ffmpeg

[Michael Niedermayer]

On Fri, Dec 19, 2008 at 03:49:16PM +0100, Reimar D?ffinger wrote:
> On Fri, Dec 19, 2008 at 02:40:57PM +0100, Michael Niedermayer wrote:
> > I dont see the problem. Let me be more concrete
> > 
> > typedef struct AVSubtitleRect {
> >     int postype;///< 0->position encoded in data (only allowed when format is text-ASS), 
> >                      1->video (0x0..1x1) relative, 
> >                      2->screen (0x0..1x1) relative, 
> >                      3->pixel relative
> >     int format; ///< Describes what is in data, 0-> bitmap, 1->ASS based text
> >     PixelFmt pix_fmt;
> >     int x;      ///< horizontal pos in 16.16 fixed point
> >     int y;      ///<   vertical pos in 16.16 fixed point
> >     int w;      ///< horizontal size in 16.16 fixed point
> >     int h;      ///<   vertical size in 16.16 fixed point
> >     uint8_t *data[4];
> >     int linesize[4]
> > } AVSubtitleRect;
> 
> Well, obviously it can be made to work (if you insist you can use
> MPlayer as a text editor...), but to me this feels like a horribly
> complex way to pass around of strings without much of an advantage.

To come back to this thread because i belive the design of subtitle
support is important, and it would be sad if a poor design was choosen
and we would just find this out once it is fully implemented or even worse
mplayer & ffmpeg would end up going seperate pathes duplicating work ...

Let me summarize what i remember from your standpoint, please correct
me if i misremember something
1. decoders should output bitmaps 
2. bitstream filters should convert betweem X->ASS and ASS->X

My suggestion
1. decoders output vector based AVSubtitleRects containing ASS or bitmaps
1b. encoders take vector based AVSubtitleRects containing ASS or bitmaps
2.  A renderer can converts ASS AVSubtitleRects to bitmaps

You say "this feels like a horribly complex way to pass around of strings
without much of an advantage", can you please elaborate on this?

The concrete problems i see with your design are
1. The current architecture is demuxer->decoder->encoder->muxer
   considering that your decoders return bitmaps its no longer possible to
   encode these to text, thus breaking the "demuxer->decoder->encoder->muxer"
2. How should mixed bitmap and text formats be represented?
   Your suggestion requires a bitstream filter to convert to ASS and then from
   ASS, but does ASS support bitmaps in every pixel format we would need, 
   besides how to put this in the char * ?
3. Does ASS support every way text can be positioned by other formats?
   I mean if we convert from X to Y the text should stay at the same
   spot on the screen given Y can represent it.

Also in the light of "horribly complex", does it not feel horribly complex
to require every ASS->X bitstream filter to be able to extract things like
position, i mean in my suggestion these would be stored in a easy accessable
struct doing the extraction just at one spot.

But what i really wanted to say is that if you can demonstrate that your
suggested design works in the general case then iam not at all against it
given someone does implement it, that is iam certainly not planningf to
reject working code, i rather doubt it will work ...

and general case here means
text -> text while not loosing effects when the destination supports the
    effects
text -> bitmaps (not a single 95% transparent screen sized bitmap)
bitmaps -> display (with bitmaps not being colorspace converted twice)
text+bitmaps -> text+bitmaps

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

While the State exists there can be no freedom; when there is freedom there
will be no State. -- Vladimir Lenin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090101/a4adc0ea/attachment.pgp>