[FFmpeg-devel] [PATCH 1/2] avcodec/qsv: add sps pps header when frame->key_frame is true

[Xiang, Haihao]

On Do, 2023-11-16 at 12:06 +0800, Zhongwei Wang via ffmpeg-devel wrote:
> The key_frame currently doesn't set extra header into frames when using qsv
> encoders.
> 
> Signed-off-by: reito <cnschwarzer at qq.com>
> ---
>  libavcodec/qsvenc.c | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
> 
> diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
> index a0144b0760..2bd85a2f91 100644
> --- a/libavcodec/qsvenc.c
> +++ b/libavcodec/qsvenc.c
> @@ -2365,6 +2365,28 @@ static int update_pic_timing_sei(AVCodecContext
> *avctx, QSVEncContext *q)
>      return updated;
>  }
>  

Failed to apply your patch via git am. Please refer to
https://ffmpeg.org/developer.html#Submitting-patches-1 about submitting patches 


> +static int set_sps_pps_encode_ctrl(AVCodecContext *avctx, const AVFrame
> *frame,
> +                               mfxEncodeCtrl *enc_ctrl)
> +{
> +    mfxExtInsertHeaders *insert_headers = NULL;
> +
> +    if (frame->key_frame) {
> +        av_log(avctx, AV_LOG_DEBUG, "Insert SPS PPS Header because of
> key_frame == 1\n");

key_frame was deprecated, you should use AV_FRAME_FLAG_KEY instead. 

> +        insert_headers = av_mallocz(sizeof(*insert_headers));
> +        if (!insert_headers)
> +            return AVERROR(ENOMEM);
> +        insert_headers->Header.BufferId = MFX_EXTBUFF_INSERT_HEADERS;
> +        insert_headers->Header.BufferSz = sizeof(*insert_headers);
> +        insert_headers->SPS = MFX_CODINGOPTION_ON;
> +        insert_headers->PPS = MFX_CODINGOPTION_ON;
> +
> +        enc_ctrl->ExtParam[enc_ctrl->NumExtParam] = (mfxExtBuffer
> *)insert_headers;
> +        enc_ctrl->NumExtParam++;

Seems MFX_EXTBUFF_INSERT_HEADERS isn't supported. I manually applied your patch,
and  got lots of warnings when running the command below:

$ ffmpeg -y -f lavfi -i testsrc=duration=100,format=nv12 -c:v h264_qsv -f null -
[...]
[h264_qsv @ 0x555acafc88f0] Warning during encoding: incompatible video
parameters (5)

Actually qsv encoders supports sps/pps insertion. First you should set frame-
>pict_type to AV_PICTURE_TYPE_I for key frames, like what ffmpeg did in
forced_kf_apply(), then you should set forced_idr option to on.

For example:
$ ffmpeg -y -f lavfi -i testsrc=duration=100,format=nv12 -g 30 -bf 0 -c:v
h264_qsv out0.mp4

frame 0, 30, 60, ... have sps/pps headers

$ ffmpeg -y -i out0.mp4 -g 60 -bf 0 -c:v h264_qsv out1.mp4

frame 0, 60, 120, ... have sps/pps headers in out1.mp4. frame 30, 90, ... don't
have sps/pps headers because they aren't key frames in out1.mp4 although they
are key frames in out0.mp4.

$ ffmpeg -y -i out0.mp4 -force_key_frames source -forced_idr on -g 60 -bf 0 -c:v
h264_qsv out2.mp4

frame 0, 30, 60, ... have sps/pps headers in out2.mp4

Thanks
Haihao


> +    }
> +
> +    return 0;
> +}
> +
>  static int encode_frame(AVCodecContext *avctx, QSVEncContext *q,
>                          const AVFrame *frame)
>  {
> @@ -2434,6 +2456,15 @@ static int encode_frame(AVCodecContext *avctx,
> QSVEncContext *q,
>          if (ret < 0)
>              goto free;
>      }
> +
> +    if ((avctx->codec_id == AV_CODEC_ID_H264 ||
> +         avctx->codec_id == AV_CODEC_ID_H265) &&
> +        enc_ctrl && QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 34)) {
> +        ret = set_sps_pps_encode_ctrl(avctx, frame, enc_ctrl);
> +        if (ret < 0)
> +            goto free;
> +    }
> +
>      if ((avctx->codec_id == AV_CODEC_ID_H264 ||
>           avctx->codec_id == AV_CODEC_ID_H265) &&
>           q->skip_frame != MFX_SKIPFRAME_NO_SKIP &&
> -- 
> 2.38.1.windows.1
> 
> 
>