[FFmpeg-devel] [PATCH] avcodec/h264_mp4toannexb: fix missing SPS/PPS to IDR frames

[Zhao Zhili]

On Tue, 2023-01-10 at 07:41 +0100, Andreas Rheinhardt wrote:
> Zhao Zhili:
> > From: Zhao Zhili <zhilizhao at tencent.com>
> > 
> > If there is a single group of SPS/PPS before an IDR frame, but no
> > SPS/PPS after that, we will miss the chance to reset
> > idr_sps_seen/idr_pps_seen. The result is missing SPS/PPS for all
> > IDR frames except the first one.
> > 
> > Signed-off-by: Zhao Zhili <zhilizhao at tencent.com>
> > ---
> >  libavcodec/h264_mp4toannexb_bsf.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/libavcodec/h264_mp4toannexb_bsf.c
> > b/libavcodec/h264_mp4toannexb_bsf.c
> > index d11be455c2..bbdeca9c44 100644
> > --- a/libavcodec/h264_mp4toannexb_bsf.c
> > +++ b/libavcodec/h264_mp4toannexb_bsf.c
> > @@ -259,7 +259,7 @@ static int h264_mp4toannexb_filter(AVBSFContext
> > *ctx, AVPacket *opkt)
> >  
> >              count_or_copy(&out, &out_size, buf, nal_size,
> >                            unit_type == H264_NAL_SPS || unit_type
> > == H264_NAL_PPS, j);
> > -            if (!new_idr && unit_type == H264_NAL_SLICE) {
> > +            if (unit_type == H264_NAL_SLICE) {
> >                  new_idr  = 1;
> >                  sps_seen = 0;
> >                  pps_seen = 0;
> 
> If I understand you correctly, you want that to include the parameter
> sets from the extradata to every IDR frame without extradata if said
> IDR
> frame follows a non-IDR frame after in-band parameter sets. There is
> just one problem with this: Said in-band parameter sets might
> overwrite
> parameter sets from extradata and inserting the outdated parameter
> sets
> from extradata lateron might make the stream undecodable.

It's a good question. Current code works when an IDR has in-band
parameter set, which is different than the out-of-band parameter set,
and the decoder doesn't miss the in-band parameter set. But there is a
high chance the decoder will miss the in-band parameter set for annexb
bitstream, e.g., playback at the middle of a live mpegts streams. It's
a serious issue of missing periodic in-band parameter set for annexb
for a lot of usecases. Handle new parameter set isn't as common as the
need of periodic in-band parameter set (I guess).

It's not a new issue that this bitstream failed to update the parameter
set, for example, it doesn't handle AV_PKT_DATA_NEW_EXTRADATA. I can
add support for
1. handling AV_PKT_DATA_NEW_EXTRADATA
2. handling in-band sps/pps

But it won't be soon.

> 
> - Andreas
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".