[FFmpeg-trac] #9650(avcodec:new): h264_qsv DECODER eventually freezes after many seeks [WINDOWS]

[FFmpeg]

#9650: h264_qsv DECODER eventually freezes after many seeks [WINDOWS]
-------------------------------------+-------------------------------------
             Reporter:  teslan       |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:  avcodec      |                  Version:  git-
             Keywords:  qsv, intel   |  master
  media sdk,                         |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:
 The h264_qsv decoder freezes after random number of seeks with flushes
 (most of the times you need max 100 seeks). Does not happen when standard
 h264 or h264_cuvid decoder is used.

 Note that this might be a bug not in FFMPEG, but in Intel Media SDK.
 I am not sure on which grave should I cry, but because there are
 developers from Intel actively enhancing the QSV support, i'll describe
 the issue both here and at Intel Media SDK github.

 Tested with both Windows-provided libmfxhw64.dll (File Version 11.21.4.14)
 and from the latest driver (File Version 21.11.3.320)

 This is the debug call stack on my decoding thread when the freeze
 happens:

 1  ZwWaitForAlertByThreadId
 ntdll                       0x7fff7a530764
 2  RtlSleepConditionVariableSRW
 ntdll                       0x7fff7a4f4021
 3  SleepConditionVariableSRW
 KERNELBASE                  0x7fff77ecce89
 4  MFXVideoVPP_GetVPPStat
 libmfxhw64                  0x7fff027ef8cc
 5  MFXVideoVPP_GetVPPStat
 libmfxhw64                  0x7fff027ec7f4
 6  MFXVideoVPP_GetVPPStat
 libmfxhw64                  0x7fff027ec69f
 7  MFXVideoVPP_GetVPPStat
 libmfxhw64                  0x7fff023575b0
 8  MFXVideoCORE_SyncOperation
 libmfxhw64                  0x7fff02337a4d
 9  av_qsv_alloc_context
 avcodec_59                  0x7fff10eb4c06
 10 av_qsv_alloc_context
 avcodec_59                  0x7fff10eb573d
 11 av_qsv_alloc_context
 avcodec_59                  0x7fff10eb59c7
 12 avcodec_default_get_buffer2
 avcodec_59                  0x7fff10a9f9b1
 13 avcodec_send_packet
 avcodec_59                  0x7fff10aa0590
 14 libavDecoder::playLoopPrivate
 libavdecoder.cpp   724      0x7ff78094f6da

 I am using FFMPEG builds from BtbN - confirmed with the latest master
 update (2022-02-15 12:35)

 How to reproduce:
 You need to flush the decoder and seek the video many times. Then after
 some decoded frames the decoder freezes. These are important excerpts from
 my code to hint how am I using it:

 Seek:
 {{{
 ...........................
 err = av_seek_frame(m.fmtctx, m.videoStream, [random-pts-in-media], 0);
         if(err != 0) {
                 av_strerror(err, errbuff, 256);
                 std::cerr << "Error Seeking. AVERROR: " << errbuff  <<
 std::endl;
         }

         ptsToFind = referencePts;
         seekMode = true;
         seekModeKeyframeFound = false;
         seekModeFlushed = false;

 .........................
 }}}
 Read&decode:

 {{{
 ...........................
  //Reads the right amount of data from stream
         err = av_read_frame(m.fmtctx, pPacket);
         if(err < 0) {
             av_strerror(err, errbuff, 256);
             std::cerr << "AV READ FRAME error or EOF " << errbuff << " "
 << m.lastPts[m.videoStream] << std::endl;
             if(err == AVERROR_EOF) {
                 emit endOfFile(m.lastPts[m.videoStream]);
             }
             av_packet_unref(pPacket);
             break;
         }

         //If we are seeking, flush the codec and reset the GOP counter (as
 we automatically found nearest I frame)
         if(seekMode) {

             if(!seekModeFlushed) {
                 if(!(pPacket->flags & AV_PKT_FLAG_KEY)){
                     av_packet_unref(pPacket);
                     continue;
                 }
                 avcodec_send_packet(avcc, NULL); //flush
                 avcodec_flush_buffers(avcc);
                 actual_gop_counter = 0;
                 seekModeFlushed = true;
             }
         }

         //Send the "right amount of data" to the decoder
         err = avcodec_send_packet(avcc, pPacket);
         if(err != 0) {
             av_strerror(err, errbuff, 256);
             std::cerr << "Error sending packet. AVERROR: " << errbuff <<
 std::endl;
             av_packet_unref(pPacket);
             continue;
         }


         //Gather first ready frame from the internal queue
         err = avcodec_receive_frame(avcc, pFrame);
         if(err != 0) {
             if(err != AVERROR(EAGAIN)) { //resource can be unavailable
 pretty often, do not write out anything in such case
                 av_strerror(err, errbuff, 256);
                 std::cerr << "Error receiving frame. AVERROR: " << errbuff
 << std::endl;
             }
             av_packet_unref(pPacket);
             av_frame_unref(pFrame);
             continue;
         }

         //Save data for average GOP computation
         if(pFrame->pict_type == AV_PICTURE_TYPE_I && actual_gop_counter !=
 0) {
             if(actual_gop_counter > max_gop) {
                 max_gop = actual_gop_counter;
             }
             actual_gop_counter = 0;
         }
         ++actual_gop_counter;

         if(seekMode) {
             if(pFrame->pts >= ptsToFind && pFrame->pts <= ptsToFind +
 avg_pts*MAX_ALLOWED_SKIP_FRAMES) {
                 seekMode = false;
                 //Because QSV is a fucked-up decoder, it always returns
 exactly TWO frames from the previous
                 //decoding chunk. And you can not flush it, it will only
 return them after you provide
                 //two fresh packets. Really fucked-up. Really. Therefore
 we can not stay with
                 //the check that if the pts is already the same or greater
 that the one we need to find.
                 //We also have to introduce if it is not too far away.
 That is defined in MAX_ALLOWED_SKIP_FRAMES
             }
             else {
                 av_frame_unref(pFrame);
                 av_packet_unref(pPacket);
                 continue; //do not proceed if still in seek
             }
         }

 .............................
 }}}





 Patches should be submitted to the ffmpeg-devel mailing list and not this
 bug tracker.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9650>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker