[FFmpeg-devel] [PATCH] matroska: Set CodecDelay and SeekPreroll in the AVStream

Michael Niedermayer michaelni at gmx.at
Tue Oct 8 21:54:01 CEST 2013


On Mon, Oct 07, 2013 at 09:05:09AM -0700, Vignesh Venkatasubramanian wrote:
> On Sun, Oct 6, 2013 at 5:19 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
> > On Thu, Oct 03, 2013 at 04:49:08PM -0700, Vignesh Venkatasubramanian wrote:
> >> On Thu, Sep 26, 2013 at 5:11 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
> >> > On Tue, Sep 24, 2013 at 02:50:16PM -0700, Vignesh Venkatasubramanian wrote:
> >> >> On Tue, Sep 24, 2013 at 2:14 PM, Michael Niedermayer <michaelni at gmx.at> wrote:
> >> >> > On Tue, Sep 24, 2013 at 12:09:31PM -0700, Vignesh Venkatasubramanian wrote:
> >> >> >> On Tue, Sep 24, 2013 at 11:36 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
> >> >> >> > On Tue, Sep 24, 2013 at 10:35:47AM -0700, Vignesh Venkatasubramanian wrote:
> >> >> >> >> On Tue, Sep 24, 2013 at 3:53 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
> >> >> >> >> > On Mon, Sep 23, 2013 at 04:03:25PM -0700, Vignesh Venkatasubramanian wrote:
> >> >> >> >> >> On Mon, Sep 23, 2013 at 4:02 PM, Vignesh Venkatasubramanian
> >> >> >> >> >> <vigneshv at google.com> wrote:
> >> >> >> >> >> > This patch exports the values of Codec Delay and Seek Preroll
> >> >> >> >> >> > container elements as in the AVStream structure. The seek_preroll
> >> >> >> >> >> > field has been added to the AVStream struct and the minor version
> >> >> >> >> >> > of libavformat has been bumped.
> >> >> >> >> >> >
> >> >> >> >> >> > Signed-off-by: Vignesh Venkatasubramanian <vigneshv at google.com>
> >> >> >> >> >> > ---
> >> >> >> >> >> >  libavformat/avformat.h    | 16 ++++++++++++++++
> >> >> >> >> >> >  libavformat/matroskadec.c | 14 ++++++++++++++
> >> >> >> >> >> >  libavformat/version.h     |  2 +-
> >> >> >> >> >> >  3 files changed, 31 insertions(+), 1 deletion(-)
> >> >> >> >> >> >
> >> >> >> >> >> > diff --git a/libavformat/avformat.h b/libavformat/avformat.h
> >> >> >> >> >> > index b18eb3f..6671a83 100644
> >> >> >> >> >> > --- a/libavformat/avformat.h
> >> >> >> >> >> > +++ b/libavformat/avformat.h
> >> >> >> >> >> > @@ -856,6 +856,9 @@ typedef struct AVStream {
> >> >> >> >> >> >
> >> >> >> >> >> >      /**
> >> >> >> >> >> >       * Number of samples to skip at the start of the frame decoded from the next packet.
> >> >> >> >> >> > +     *
> >> >> >> >> >> > +     * Code outside avformat should access this field using:
> >> >> >> >> >> > +     * av_stream_get_skip_samples/set_skip_samples(stream)
> >> >> >> >> >> >       */
> >> >> >> >> >> >      int skip_samples;
> >> >> >> >> >
> >> >> >> >> > when does code outside lavf need to access this ?
> >> >> >> >> >
> >> >> >> >>
> >> >> >> >> you are right. it need not be accessed outside lavf.
> >> >> >> >>
> >> >> >> >> >
> >> >> >> >> >> >
> >> >> >> >> >> > @@ -888,10 +891,23 @@ typedef struct AVStream {
> >> >> >> >> >> >       */
> >> >> >> >> >> >      int pts_wrap_behavior;
> >> >> >> >> >> >
> >> >> >> >> >> > +    /**
> >> >> >> >> >> > +     * Number of samples to skip after a discontinuity. For example, when a seek
> >> >> >> >> >> > +     * happens.
> >> >> >> >> >> > +     *
> >> >> >> >> >> > +     * Code outside avformat should access this field using:
> >> >> >> >> >> > +     * av_stream_get/set_seek_preroll(stream)
> >> >> >> >> >> > +     */
> >> >> >> >> >> > +    int seek_preroll;
> >> >> >> >> >> > +
> >> >> >> >> >> >  } AVStream;
> >> >> >> >> >> >
> >> >> >> >> >> >  AVRational av_stream_get_r_frame_rate(const AVStream *s);
> >> >> >> >> >> >  void       av_stream_set_r_frame_rate(AVStream *s, AVRational r);
> >> >> >> >> >> > +int  av_stream_get_skip_samples(const AVStream *s);
> >> >> >> >> >> > +void av_stream_set_skip_samples(AVStream *s, int skip_samples);
> >> >> >> >> >> > +int  av_stream_get_seek_preroll(const AVStream *s);
> >> >> >> >> >> > +void av_stream_set_seek_preroll(AVStream *s, int seek_preroll);
> >> >> >> >> >> >
> >> >> >> >> >> >  #define AV_PROGRAM_RUNNING 1
> >> >> >> >> >> >
> >> >> >> >> >> > diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c
> >> >> >> >> >> > index a1b7f56..ecbc723 100644
> >> >> >> >> >> > --- a/libavformat/matroskadec.c
> >> >> >> >> >> > +++ b/libavformat/matroskadec.c
> >> >> >> >> >> > @@ -163,6 +163,8 @@ typedef struct {
> >> >> >> >> >> >      uint64_t default_duration;
> >> >> >> >> >> >      uint64_t flag_default;
> >> >> >> >> >> >      uint64_t flag_forced;
> >> >> >> >> >> > +    uint64_t codec_delay;
> >> >> >> >> >> > +    uint64_t seek_preroll;
> >> >> >> >> >> >      MatroskaTrackVideo video;
> >> >> >> >> >> >      MatroskaTrackAudio audio;
> >> >> >> >> >> >      MatroskaTrackOperation operation;
> >> >> >> >> >> > @@ -410,6 +412,8 @@ static EbmlSyntax matroska_track[] = {
> >> >> >> >> >> >      { MATROSKA_ID_TRACKOPERATION,       EBML_NEST, 0, offsetof(MatroskaTrack,operation), {.n=matroska_track_operation} },
> >> >> >> >> >> >      { MATROSKA_ID_TRACKCONTENTENCODINGS,EBML_NEST, 0, 0, {.n=matroska_track_encodings} },
> >> >> >> >> >> >      { MATROSKA_ID_TRACKMAXBLKADDID,     EBML_UINT, 0, offsetof(MatroskaTrack,max_block_additional_id) },
> >> >> >> >> >> > +    { MATROSKA_ID_CODECDELAY,           EBML_UINT, 0, offsetof(MatroskaTrack,codec_delay) },
> >> >> >> >> >> > +    { MATROSKA_ID_SEEKPREROLL,          EBML_UINT, 0, offsetof(MatroskaTrack,seek_preroll) },
> >> >> >> >> >> >      { MATROSKA_ID_TRACKFLAGENABLED,     EBML_NONE },
> >> >> >> >> >> >      { MATROSKA_ID_TRACKFLAGLACING,      EBML_NONE },
> >> >> >> >> >> >      { MATROSKA_ID_CODECNAME,            EBML_NONE },
> >> >> >> >> >
> >> >> >> >> >> > @@ -1874,6 +1878,16 @@ static int matroska_read_header(AVFormatContext *s)
> >> >> >> >> >> >              st->codec->bits_per_coded_sample = track->audio.bitdepth;
> >> >> >> >> >> >              if (st->codec->codec_id != AV_CODEC_ID_AAC)
> >> >> >> >> >> >              st->need_parsing = AVSTREAM_PARSE_HEADERS;
> >> >> >> >> >> > +            if (track->codec_delay > 0) {
> >> >> >> >> >> > +                st->skip_samples = av_rescale_q(track->codec_delay,
> >> >> >> >> >> > +                                                (AVRational){1, 1000000000},
> >> >> >> >> >> > +                                                (AVRational){1, st->codec->sample_rate});
> >> >> >> >> >> > +            }
> >> >> >> >> >
> >> >> >> >> > i suspect this alone wont work when the user seeks back to the first
> >> >> >> >> > packet
> >> >> >> >>
> >> >> >> >> i don't understand why. can you please explain?
> >> >> >> >
> >> >> >> > because its set when parsing the header but when seeking back the
> >> >> >> > header is not re parsed so its not set again so the first packet
> >> >> >> > would then skip a different amount
> >> >> >> > thats unless iam missing something
> >> >> >> >
> >> >> >>
> >> >> >> yes, you are right. it is up to the decoder implementation to make sure
> >> >> >> that it uses the correct skip_samples value when the input timestamp is
> >> >> >> zero (or start of stream) when seeking.
> >> >> >
> >> >> > the decoder doesnt know when the timestamp is zero, a file might start
> >> >> > from a timestamp differnt than zero
> >> >> >
> >> >>
> >> >> i think i haven't communicated correctly about what i'm trying to
> >> >> achieve with this patch.
> >> >> basically, lavf will put this container value in the stream struct
> >> >> while parsing the
> >> >> header. once it populated, this field should not change (on seeking,
> >> >> etc.). so , it's a place
> >> >> where i can put the container field in the stream.
> >> >
> >> > AVStream->skip_samples will simply cause side data to be added to the
> >> > next packet that indicate to the decoder that a specific number of
> >> > samples should be skiped. skip_samples will be reset to 0 after it
> >> > has been used in this way.
> >> >
> >> > See the code in utils.c that uses skip_samples, its just 5 lines
> >> >
> >> >
> >> > A demuxer could just directly generate the sidedata but that is
> >> > probably often more complex implementation wise
> >> >
> >>
> >> Here is what I am trying to achieve: I need to use libavformat's
> >> demuxer to demux the file but not use libavcodec for the decoding. So
> >
> > that shouldnt affect how values are exported from libavformat
> >
> >
> >> in that case, i need libavformat to export this codec delay container
> >> element's value somewhere. I presumed that to be the AVStream
> >> structure. Can you please tell me how to do this if not through
> >> AVStream?
> >
> > there are many ways on how to export values, AVStream, AVCodecContext
> > AVPacket side data, metadata, ...
> > Each of these have different limitations
> > AVStream/AVCodecContext when used to communicate with the outside
> > should not hold data that changes mid stream (any packet queue
> > would make matching them to packets hard)
> > When the demuxer and decoder layer need to keep track of different
> > values then both AVStream and AVCodecContext might each have the same
> > field like sample_aspect_ratio.
> > Also the decoder/encoder does not have direct access to AVStream
> > But the demuxer/muxer do have access to AVCodecContext, though this
> > is not guranteed to be the very same struct that is used by the
> > actual codec.
> >
> >
> > also
> > what i think is important is that its consistent between muxer and
> > demuxer. Also existing fields should be used when theres no reason
> > not to use them but fields should also not be misused.
> > AVStream->skip_samples is intended to transport values from a demuxers
> > header parser to the first AVPackets sidedata. Currently
> > that requires the value to be set again when seeking to the begin.
> > That could be changed of course if it simplifies something ...
> >
> > Now, above does not awnser your question what to use, but rather
> > tries to explain the differences of the various options.
> > I have no real preferrance what to use and i dont know the fine
> > details of the matroska fields and the specification is kind of not
> > explaining them very well either
> >
> 
> Based on your response, i now have a better understanding of the
> various ways to export this information from libavformat. I feel that
> exporting seek_preroll this way is fine. Regarding codec_delay, since
> it is a per file metadata and not associated with a packet, I feel
> that it should not go into skip_samples (which essentially modifies
> the next packet, rather than exporting it as a per-file metadata) but
> a new field should be created in AVStream instead. Please let me know
> if you think this behavior is good so that I can go ahead and do the
> patches.

can you explain how this will work on the encoder -> muxer side ?


> 
> >
> >>
> >> The same goes for the other SeekPreRoll element too.
> >>
> >> Thanks!
> >>
> >> > [...]
> >> > --
> >> > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >> >
> >> > No snowflake in an avalanche ever feels responsible. -- Voltaire
> >> >
> >> > _______________________________________________
> >> > ffmpeg-devel mailing list
> >> > ffmpeg-devel at ffmpeg.org
> >> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >> >
> >> _______________________________________________
> >> ffmpeg-devel mailing list
> >> ffmpeg-devel at ffmpeg.org
> >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >>
> >
> > --
> > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >
> > If you think the mosad wants you dead since a long time then you are either
> > wrong or dead since a long time.
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No human being will ever know the Truth, for even if they happen to say it
by chance, they would not even known they had done so. -- Xenophanes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20131008/8ca7b828/attachment.asc>


More information about the ffmpeg-devel mailing list