[FFmpeg-devel] [PATCH] Add flag to drop non-forced subtitle

Michael Niedermayer michaelni at gmx.at
Sun Jun 29 20:21:29 CEST 2014


On Sun, Jun 29, 2014 at 05:36:32PM +0200, Oliver Fromme wrote:
> Michael Niedermayer wrote:
>  > On Mon, May 26, 2014 at 12:03:35PM +0200, Oliver Fromme wrote:
>  > > Michael Niedermayer wrote:
>  > > > On Thu, May 22, 2014 at 07:39:11PM +0200, Oliver Fromme wrote:
>  > > > > Hello,
>  > > > > 
>  > > > > As I mentioned earlier this week, I'm trying to improve support
>  > > > > for forced and partially forced subtitles.  The patch below is
>  > > > > the next step.
>  > > > > 
>  > > > > The patch changes four files.  I'll explain in detail what the
>  > > > > changes are good for:
>  > > > > 
>  > > > > libavcodec/avcodec.h -- Introduce a new codec flag for subtitle
>  > > > > encoders, it's called CODEC_FLAG_FORCED_SUBS.  When set, this
>  > > > > flag indicates that the encoder should write only such subtitle
>  > > > > frames that are marked as forced.  Frames that are *not* forced
>  > > > > will be ignored by the encoder.
>  > > > > 
>  > > > > By the way, is there a rule for allocating bit masks for new
>  > > > > flags?  I just took the first unused one, that is 0x0080.
>  > > > > 
>  > > > > libavcodec/options_table.h -- Add an option so the new flag can
>  > > > > be set on the command line like this:  -flags +forced_subs
>  > > > > 
>  > > > > Before I continue, I need to explain some background:  A single
>  > > > > subtitle frame can consist of multiple rectangles, each one has
>  > > > > its own forced flag.  PGS (i.e. the Blu-Ray subtitle format)
>  > > > > supports multiple rectangles, but VOBSUB (a.k.a. dvdsub) does
>  > > > > not.  So, if we have a subtitle frame with multiple rectangles,
>  > > > > we have to merge them into one rectangle when encoding to dvdsub.
>  > > > > The dvdsubenc encoder already does that, but it doesn't respect
>  > > > > the forced flag yet.  Now comes my patch ...
>  > > > > 
>  > > > > libavcodec/dvdsubenc.c -- If CODEC_FLAG_FORCED_SUBS is set and
>  > > > > there are multiple rectangles, we need to figure out which ones
>  > > > > are forced and which ones are not, and then merge only the ones
>  > > > > that are forced.  The new variable "first" contains the index
>  > > > > of the first rectangle that needs to be merged (otherwise it
>  > > > > contains 0, so the behaviour is unchanged if the codec flag is
>  > > > > not set).
>  > > > > 
>  > > > > In all the following loops that iterate over the rectangles,
>  > > > > any rectangles that are non-forced are skipped when the codec
>  > > > > flag is set.  That's all.
>  > > > > 
>  > > > > Unfortunately, there's a special case that cannot be handled
>  > > > > here in the encoder:  When *all* rectangles of this subtitle
>  > > > > are non-forced (and the codec flag is set).  In this case we
>  > > > > would have to skip all rectangles and produce an empty frame.
>  > > > > This is not possible in the encoder ...  The API expects that
>  > > > > the codec's encode() function produces a frame for every input
>  > > > > frame.  Therefore, this case has to be handled at a higher
>  > > > > level:
>  > > > > 
>  > > > > ffmpeg.c -- In the do_subtitle_out() function, we check if
>  > > > > CODEC_FLAG_FORCED_SUBS is set.  If it is, we check if *all*
>  > > > > of the rectangles of the current subtitle frame are non-forced.
>  > > > > If that's the case, we return right away, not calling the
>  > > > > encoder at all, not calling write_frame(), not increasing the
>  > > > > frame counter etc.
>  > > > > 
>  > > > > You can test the new functionality with the file that I had
>  > > > > uploaded on May 18th, called subt-forced-test.mkv.  It
>  > > > > contains one dvdsub subtitle track that has 991 frames, 2 of
>  > > > > which are forced.
>  > > > > For example, the following command could be used to "split"
>  > > > > the subtitles in two output files:  The first one contains
>  > > > > all of them, the second one contains only the forced frames:
>  > > > > 
>  > > > >     $ ffmpeg -i subt-forced-test.mkv \
>  > > > >             -codec:s dvdsub test1.mkv \
>  > > > >             -codec:s dvdsub -flags:s +forced_subs test2.mkv
>  > > > > 
>  > > > >     $ mkvextract tracks test1.mkv 0:test1.idx
>  > > > >     $ mkvextract tracks test2.mkv 0:test2.idx
>  > > > >     $ grep -c timestamp test[12].idx
>  > > > >     test1.idx:991
>  > > > >     test2.idx:2
>  > > > > 
>  > > > > Voila.
>  > > > > 
>  > > > > Best regards
>  > > > >    Oliver
>  > > > > 
>  > > > > 
>  > > > > --- ffmpeg/libavcodec/avcodec.h.orig	2014-05-13 19:20:05.000000000 +0200
>  > > > > +++ ffmpeg/libavcodec/avcodec.h	2014-05-21 18:52:28.000000000 +0200
>  > > > > @@ -758,6 +758,7 @@
>  > > > >   */
>  > > > >  #define CODEC_FLAG_MV0    0x0040
>  > > > >  #endif
>  > > > > +#define CODEC_FLAG_FORCED_SUBS     0x0080   ///< Encode forced subtitles only.
>  > > > >  #if FF_API_INPUT_PRESERVED
>  > > > >  /**
>  > > > >   * @deprecated passing reference-counted frames to the encoders replaces this
>  > > > > --- ffmpeg/libavcodec/options_table.h.orig	2014-05-13 19:20:05.000000000 +0200
>  > > > > +++ ffmpeg/libavcodec/options_table.h	2014-05-21 18:55:15.000000000 +0200
>  > > > > @@ -60,6 +60,7 @@
>  > > > >  #if FF_API_MV0
>  > > > >  {"mv0", "always try a mb with mv=<0,0>", 0, AV_OPT_TYPE_CONST, {.i64 = CODEC_FLAG_MV0 }, INT_MIN, INT_MAX, V|E, "flags"},
>  > > > >  #endif
>  > > > > +{"forced_subs", "encode forced subtitle captions only", 0, AV_OPT_TYPE_CONST, {.i64 = CODEC_FLAG_FORCED_SUBS }, INT_MIN, INT_MAX, S|E, "flags"},
>  > > > >  #if FF_API_INPUT_PRESERVED
>  > > > >  {"input_preserved", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = CODEC_FLAG_INPUT_PRESERVED }, INT_MIN, INT_MAX, 0, "flags"},
>  > > > >  #endif
>  > > > 
>  > > > > --- ffmpeg/libavcodec/dvdsubenc.c.orig	2014-05-16 23:35:29.000000000 +0200
>  > > > > +++ ffmpeg/libavcodec/dvdsubenc.c	2014-05-21 20:03:55.000000000 +0200
>  > > > 
>  > > > why is this done in dvdsubenc and not utils.c for all subtitle
>  > > > encoders ?
>  > > 
>  > > utils.c has no knowledge if the subtitle codec supports 
>  > > multiple rectangles per subtitle frame.  dvdsub does not,
>  > > so dvdsubenc has to merge multiple rectangles into one
>  > > frame, and also handle per-rectangle forced flags.
>  > 
>  > i dont understand the problem you describe
>  > 
>  > your patch adds a flag that says "encode forced subtitle captions only"
>  > so it should be possible to drop non forced rectangles before the
>  > encoder in utils.c, what am i missing ?
> 
> First of all, I'm sorry for the late reply.  I was swamped with
> other work and private things, and had near zero spare time.
> 
> I agree with you that it is not optimal to filter the rectangles
> in the dvdsub encoder.  I'm not very happy with this either.
> It would be better to do that in one place for all encoders
> (even though dvdsubenc is currently ffmpeg's only encoder for
> graphical subtitles).
> 

> The problem with dropping rectangles in utils.c is that there
> might be multiple output streams.  The flag ("forced subtitles
> only") might be set for some output streams, but not for others.
> So, if we dropped some rectangles from the AVSubtitle structure
> in utils.c, they will be missing from other output streams that
> are encoded afterwards.

You can put the, to be kept rectangles in a new AVSubtitle struct.

what does this need ? 11 lines of code ?
if (only forced) {
    AVSubtitle newsub = *sub;
    newsub.num_rects = 0;
    newsub.rects = av_malloc_array(sub->num_rects, sizeof(newsub.rects*));
    for ()
        if (forced)
            newsub.rects[newsub.num_rects++] = sub->rects[i];

    encode newsub

    av_freep(&newsub->rects);
} else
    encode sub

    
> 
> On the other hand, the dvdsub encoder has to merge rectangles
> into one frame, so it has to have a loop for them anyway, so it
> was very easy to add the new feature there.  Basically it's just
> one "if" instruction in the loops that iterate the rectangles.
> That's why I chose this solution.

its not 1 but 4 loops you have to modify that way and the feature
would be unavailable to all subtitle encoders except dvdsubs
and i dont see why doing this in utils.c wouldnt be simpler as well


> 
> If you have a better idea how to solve the problem, please let
> me know.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Frequently ignored answer#1 FFmpeg bugs should be sent to our bugtracker. User
questions about the command line tools should be sent to the ffmpeg-user ML.
And questions about how to use libav* should be sent to the libav-user ML.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140629/9c213c9f/attachment.asc>


More information about the ffmpeg-devel mailing list