[FFmpeg-devel] [PATCH v5] Add SUP/PGS subtitle demuxer
Petri Hintukainen
phintuka at gmail.com
Wed Sep 3 13:31:42 CEST 2014
On ma, 2014-09-01 at 16:40 +0200, wm4 wrote:
> On Mon, 01 Sep 2014 13:37:23 +0300
> Petri Hintukainen <phintuka at gmail.com> wrote:
>
> > On ma, 2014-09-01 at 10:05 +0200, Hendrik Leppkes wrote:
> > > On Mon, Sep 1, 2014 at 10:01 AM, Petri Hintukainen <phintuka at gmail.com> wrote:
> > > > On ma, 2014-09-01 at 00:34 +0200, wm4 wrote:
> > > >> ---
> > > >> Use AV_RB16 instead of memcpy.
> > > >> Don't use AVPROBE_SCORE_EXTENSION.
> > > >> ---
> > > >> libavformat/Makefile | 1 +
> > > >> libavformat/allformats.c | 1 +
> > > >> libavformat/supdec.c | 107 +++++++++++++++++++++++++++++++++++++++++++++++
> > > >> 3 files changed, 109 insertions(+)
> > > >> create mode 100644 libavformat/supdec.c
> > > >>
> > > >> diff --git a/libavformat/Makefile b/libavformat/Makefile
> > > >> index 3d124fb..b4965fe 100644
> > > >> --- a/libavformat/Makefile
> > > >> +++ b/libavformat/Makefile
> > > >> @@ -405,6 +405,7 @@ OBJS-$(CONFIG_SRT_MUXER) += srtenc.o
> > > >> OBJS-$(CONFIG_STR_DEMUXER) += psxstr.o
> > > >> OBJS-$(CONFIG_SUBVIEWER1_DEMUXER) += subviewer1dec.o subtitles.o
> > > >> OBJS-$(CONFIG_SUBVIEWER_DEMUXER) += subviewerdec.o subtitles.o
> > > >> +OBJS-$(CONFIG_SUP_DEMUXER) += supdec.o
> > > >> OBJS-$(CONFIG_SWF_DEMUXER) += swfdec.o swf.o
> > > >> OBJS-$(CONFIG_SWF_MUXER) += swfenc.o swf.o
> > > >> OBJS-$(CONFIG_TAK_DEMUXER) += takdec.o apetag.o img2.o rawdec.o
> > > >> diff --git a/libavformat/allformats.c b/libavformat/allformats.c
> > > >> index 8f70c4b..e6c0e5f 100644
> > > >> --- a/libavformat/allformats.c
> > > >> +++ b/libavformat/allformats.c
> > > >> @@ -280,6 +280,7 @@ void av_register_all(void)
> > > >> REGISTER_DEMUXER (STR, str);
> > > >> REGISTER_DEMUXER (SUBVIEWER1, subviewer1);
> > > >> REGISTER_DEMUXER (SUBVIEWER, subviewer);
> > > >> + REGISTER_DEMUXER (SUP, sup);
> > > >> REGISTER_MUXDEMUX(SWF, swf);
> > > >> REGISTER_DEMUXER (TAK, tak);
> > > >> REGISTER_MUXER (TEE, tee);
> > > >> diff --git a/libavformat/supdec.c b/libavformat/supdec.c
> > > >> new file mode 100644
> > > >> index 0000000..3726c97
> > > >> --- /dev/null
> > > >> +++ b/libavformat/supdec.c
> > > >> @@ -0,0 +1,107 @@
> > > >> +/*
> > > >> + * This file is part of FFmpeg.
> > > >> + *
> > > >> + * FFmpeg is free software; you can redistribute it and/or
> > > >> + * modify it under the terms of the GNU Lesser General Public
> > > >> + * License as published by the Free Software Foundation; either
> > > >> + * version 2.1 of the License, or (at your option) any later version.
> > > >> + *
> > > >> + * FFmpeg is distributed in the hope that it will be useful,
> > > >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > > >> + * Lesser General Public License for more details.
> > > >> + *
> > > >> + * You should have received a copy of the GNU Lesser General Public
> > > >> + * License along with FFmpeg; if not, write to the Free Software
> > > >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> > > >> + */
> > > >> +
> > > >> +#include "avformat.h"
> > > >> +#include "internal.h"
> > > >> +#include "libavutil/intreadwrite.h"
> > > >> +
> > > >> +#define SUP_PGS_MAGIC 0x5047 /* "PG", big endian */
> > > >> +
> > > >> +static int sup_read_header(AVFormatContext *s)
> > > >> +{
> > > >> + AVStream *st = avformat_new_stream(s, NULL);
> > > >> + if (!st)
> > > >> + return AVERROR(ENOMEM);
> > > >> + st->codec->codec_type = AVMEDIA_TYPE_SUBTITLE;
> > > >> + st->codec->codec_id = AV_CODEC_ID_HDMV_PGS_SUBTITLE;
> > > >> + avpriv_set_pts_info(st, 32, 1, 90000);
> > > >> +
> > > >> + return 0;
> > > >> +}
> > > >> +
> > > >> +static int sup_read_packet(struct AVFormatContext *s, AVPacket *pkt)
> > > >> +{
> > > >> + int64_t pts, pos;
> > > >> + int ret;
> > > >> +
> > > >> + pos = avio_tell(s->pb);
> > > >> +
> > > >> + if (avio_rb16(s->pb) != SUP_PGS_MAGIC)
> > > >> + return avio_feof(s->pb) ? AVERROR_EOF : AVERROR_INVALIDDATA;
> > > >> +
> > > >> + pts = avio_rb32(s->pb);
> > > >> + avio_rb32(s->pb); /* discard DTS (usually 0, and useless) */
> > > >
> > > > Is there some reason to discard DTS ? I would keep DTS when it is
> > > > present, even if ffmpeg decoder does not use it. There are several
> > > > reasons why DTS should be kept:
> > > >
> > >
> > > Setting a DTS of 0 for all frames could result in serious problems.
> >
> > Unless it is set only when > 0 ?
>
> Can we assume this?
I mean
dts = avio_rb32(s->pb);
if (dts != 0) {
pkt->dts = dts;
}
should be safe compromise between protecting against "broken" files and
preserving the timestamps ?
Also, palette segments have no DTS. I think such segments have DTS = 0
in .sup header.
> Assuming you never can have the same DTS values, you could read 2
> packets at start. If both have DTS=0, you know that DTS should be
> discarded.
Not necessarily. DTS of every segment in display set can be the same.
Decoding and compositing imply some rules for timestamps, and we could
calculate upper limit for segments that need to be probed until DTS > 0
must appear. I just don't know if it is worth of the work (I don't know
if (.m2ts) files with DTS=0 even exist in real life).
> But that sounds like additional effort. Is there a way to
> make libavformat/utils.c do this?
>
> All in all, it might just be best to discard DTS if it's 0.
>
> Suggestions welcome.
>
> Yes, we can't pass through DTS if it's all set to 0, it would break too
> much.
>
> > > Shouldn't DTS and PTS be the same for subtitles anyway,
> >
> > The stream does not carry subtitles, it carries PGS (Presentation
> > Graphics Stream) segments. Each segment has separate PTS/DTS timestamps.
> > Subtitle is a result of applying composition rules from two or more
> > Presentation Composition Segments to the data carried in other
> > segments.
> >
> > > considering they are not re-ordered?
> >
> > In PGS stream DTS is monotonic but PTS is not. Segments are (kind of)
> > re-ordered after decoding. Here are timestamps of the segments used to
> > compose two first subtitles in #2208 sample (00038.m2ts):
> >
> > First subtitle:
> > SEG 0x16 DTS 000249146 PTS 000255255 // PCS
> > SEG 0x17 DTS 000249146 PTS 000254978 // WDS
> > SEG 0x14 DTS 000249146 PTS 000249146 // PDS
> > SEG 0x15 DTS 000249146 PTS 000249699 // ODS
> > SEG 0x80 DTS 000249699 PTS 000249699 // EOD
> >
> > Hide first subtitle:
> > SEG 0x16 DTS 000607829 PTS 000608107 // PCS
> > SEG 0x17 DTS 000607829 PTS 000607830 // WDS
> > SEG 0x80 DTS 000607829 PTS 000607829 // EOD
> >
> > Second subtitle:
> > SEG 0x16 DTS 000609547 PTS 000615615 // PCS
> > SEG 0x17 DTS 000609547 PTS 000615379 // WDS
> > SEG 0x14 DTS 000609547 PTS 000609547 // PDS
> > SEG 0x15 DTS 000609547 PTS 000610019 // ODS
> > SEG 0x80 DTS 000610019 PTS 000610019 // EOD
> >
> > Hide second subtitle:
> > SEG 0x16 DTS 000743005 PTS 000743242 // PCS
> > SEG 0x17 DTS 000743005 PTS 000743006 // WDS
> > SEG 0x80 DTS 000743005 PTS 000743005 // EOD
>>
> > Each segment has separate timestamps. PTS of Presentation Composition
> > Segment is the time when subtitle is displayed or hidden. PTS in
> > palette / object segments tells when the segment is valid (and available
> > to be used in compositing).
> > First segment in the stream is presentation composition segment.
> > Following window, palette, object... segments must have smaller PTS or
> > those could not be used when compositing the image.
> >
> > The format allows quite complex animated presentation graphics at full
> > HD resolution. Muxing the stream based on PTS only could be fatal for
> > resource-limited HW player. DTS can also be important to avoid exceeding
> > max. bit rate when muxing.
>
> Do you have a sample that clearly shows failure if correct DTS handling
> is ignored?
No, and I don't expect missing DTS being big problem for software
players (except XBMC). But, DTS is there for a reason, dropping it
without any reason seems wrong. Muxing PG stream back to .m2ts requires
correctly calculated DTS timestamps, or the stream is not BluRay
compatible.
I don't know how HW players behave when decoding / presentation
deadlines are missed. Some may just show the images late (?), others
might drop those (?). I also don't know what happens when decoding
requires more HW resources than there are available.
If we do not set DTS in demuxer, will ffmpeg generate it at some later
stage before writing to output file ? If not, will non-monotonic PTS
cause any problems with ffmpeg ? If DTS is generated from PTS, it is
most likely incorrect.
> How does it work when embedded in formats that don't have DTS, like
> Matroska or eac3to .sup files?
Matroska (mkvmerge) is even worse, it merges all segments of display set
to single matroska packet, losing all timestamps expect the PTS of
presentation segment. (maybe to prevent seeking to middle of display
set ?).
If someone wants to remux such stream back to .m2ts file, all missing
timestamps must be recalculated (or the resulting stream is not BluRay
compatible).
- Petri
More information about the ffmpeg-devel
mailing list