[FFmpeg-trac] #9766(undetermined:new): h264_qsv decoder does not work with RTSP

[FFmpeg]

#9766: h264_qsv decoder does not work with RTSP
-------------------------------------+-------------------------------------
             Reporter:  Steve        |                     Type:  defect
  Sampson                            |
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:  git-
  undetermined                       |  master
             Keywords:  QSV RTSP     |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:
 How to reproduce:
 {{{
 % ffmpeg.exe -init_hw_device qsv:hw,child_device_type=d3d11va -hwaccel qsv
 -c:v h264_qsv -hwaccel_output_format qsv -i
 rtsp://user:password@172.16.1.105/axis-media/media.amp -f null -

 ffmpeg version: ffmpeg version 5.0-full_build-www.gyan.dev downloaded on
 2022-01-16, so build is around that date.

 Running on Windows 10 with an Intel GPU (Intel UHD Graphics 620, but I
 suspect this problem isn't related to the exact hardware version - it also
 reproduced on another machine). The RTSP stream comes from a camera. It
 looks like it has a pixel format of 720x576, pix_fmt: yuvj420p. If there
 is a way to capture this stream such that someone else can reproduce the
 error, please let me know. I suspect the reproduction of this problem
 requires a genuine RTSP server.

 When running, it initializes the connection to the RTSP server and then
 logs an error

   Error initializing the MFX video decoder: invalid video parameters (-15)

 The same command line successfully decodes using an mp4 file instead of an
 RTSP input (captured from the camera using   ffmpeg -i
 rtsp://user:password@172.16.1.105/axis-media/media.amp -acodec copy
 -vcodec copy c:/abc.mp4)

 ''The RTSP stream decodes successfully without any hardware
 acceleration.'' That is, with a command line of "ffmpeg.exe -i
 rtsp://user:password@172.16.1.105/axis-media/media.amp -f null -". It also
 is correctly displayed in VLC. I tried many different combinations of
 command line parameters (for example just with -hwaccel qsv -c:v h264_qsv)
 and none were successful.

 }}}

 My commentary/analysis
 {{{
 Note that I am not at all an expert on FFMPEG. I was tasked to GPU-enable
 our existing code and this is my first serious look at FFMPEG.

 The use case is to display the video from a set of cameras (all RTSP, all
 H264). Our code uses the FFMPEG API and doesn't use the FFMPEG program but
 my tests have been using FFMPEG.

 The actual build I wish(ed) to use is an older build of ffmpeg (branched
 in 2019 and slightly modified by my team - but the decoding wasn't
 touched). It gives a different error string but it may be a similar root
 cause. I tried to debug it to figure out what was going wrong. The error
 in that version was "non-existing PPS 0 referenced" (from parse_nal_units)
 and then "Decoding pixel format '(null)' is not supported" (from
 ff_qsv_process_data).

 I can't easily make my own build of FFMPEG (the process of building with
 Windows is too complex for me to quickly do) or I would attempt using the
 latest code.

 The SPD returned by the camera is the following

 SDP:
 v=0
 o=- 1651685682795324 1651685682795324 IN IP4 172.16.1.105
 s=Media Presentation
 e=NONE
 b=AS:50000
 t=0 0
 a=control:*
 a=range:npt=0.000000-
 m=video 0 RTP/AVP 96
 c=IN IP4 0.0.0.0
 b=AS:50000
 a=framerate:25.0
 a=transform:0.937500,0.000000,0.000000;0.000000,1.000000,0.000000;0.000000,0.000000,1.000000
 a=control:trackID=1
 a=rtpmap:96 H264/90000
 a=fmtp:96 packetization-mode=1; profile-level-id=420029; sprop-parameter-
 sets=Z0IAKeKQFoJNgScFAQXh4kRU,aM48gA==
 }}}
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9766>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker