[FFmpeg-trac] #9713(avcodec:new): Hardware accelerated decoding fails on M1 macs for certain videos encoded with h264

[FFmpeg]

#9713: Hardware accelerated decoding fails on M1 macs for  certain videos encoded
with h264
-------------------------------------+-------------------------------------
             Reporter:  mikffmpeg    |                    Owner:  (none)
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avcodec
              Version:  git-master   |               Resolution:
             Keywords:  avcodec;     |               Blocked By:
  m1; mac; h264                      |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Description changed by mikffmpeg:

Old description:

> '''Summary of the bug:'''
> Using hardware acceleration (i.e. videotoolbox) to decode the frames of
> some video streams encoded with h264 fails on recent M1 mac computers.
>
> '''How to reproduce:'''
> Ensure you are using a Mac with an Apple M1 chip.
> Download the attached sample file and use ffmpeg to target it with
> {{{
> % ffmpeg -hwaccel videotoolbox -i m1_not_decodable_with_acceleration.ts
> -an output.ts
> }}}
> The terminal will then repeatedly print
> {{{
> [h264 @ 0x7f85b5f32ec0] hardware accelerator failed to decode picture
> Error while decoding stream #0:0: Unknown error occurred
> }}}
>
> '''Other details:'''
> This bug does not occur with all videos encoded with h264. Although the
> issue can be reproduced with the most recent ffmpeg binary (version
> N-106494-g1291568c98-tessus), when looking at the source code of FFmpeg
> 3.3, it seems that the failure is caused by a call to
> [https://developer.apple.com/documentation/videotoolbox/1536071-vtdecompressionsessiondecodefram?language=objc/
> VTDecompressionSessionDecodeFrame]. This call does not return an error
> code, but it alters the videotoolbox context's state so as not to fail on
> a null pointer check.

New description:

 '''Summary of the bug:'''
 Using hardware acceleration (i.e. videotoolbox) to decode the frames of
 some video streams encoded with h264 fails on recent M1 mac computers.

 '''How to reproduce:'''
 Ensure you are using a Mac with an Apple M1 chip.
 Download the attached sample file and use ffmpeg to target it with
 {{{
 % ffmpeg -hwaccel videotoolbox -i m1_not_decodable_with_acceleration.ts
 -an output.ts
 }}}
 The terminal will then repeatedly print
 {{{
 [h264 @ 0x7f85b5f32ec0] hardware accelerator failed to decode picture
 Error while decoding stream #0:0: Unknown error occurred
 }}}

 '''Other details:'''
 This bug does not occur with all videos encoded with h264.

 Although the issue can be reproduced with the most recent ffmpeg binary
 (version N-106494-g1291568c98-tessus), when looking at the source code of
 FFmpeg 3.3, it seems that the failure is caused by a call to
 [https://developer.apple.com/documentation/videotoolbox/1536071-vtdecompressionsessiondecodefram?language=objc/
 VTDecompressionSessionDecodeFrame]. When decoding works, this call alters
 the videotoolbox context's state so as to populate a 'frame' field. When
 decoding fails, the call does not return an error code, but it also does
 not change the videotoolbox context. The result is that 'frame' stays
 null, which then triggers a null pointer check down the line.

--
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9713#comment:2>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker