[FFmpeg-trac] #6379(avcodec:new): vaapi_encode_check_config invalid free

[FFmpeg]

#6379: vaapi_encode_check_config invalid free
-------------------------------------+-----------------------------------
             Reporter:  serafean     |                    Owner:
                 Type:  defect       |                   Status:  new
             Priority:  important    |                Component:  avcodec
              Version:  3.2.4        |               Resolution:
             Keywords:  crash vaapi  |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-----------------------------------

Comment (by serafean):

 ffmpeg version : N-85962-g164e277
 valgrind still raises SIGILL
 MALLOC_CHECK_=2 still crashes with
 {{{
 (gdb) bt
 #0  0x00007ffabdfc6eb8 in __GI_raise (sig=sig at entry=6) at
 ../sysdeps/unix/sysv/linux/raise.c:54
 #1  0x00007ffabdfc844a in __GI_abort () at abort.c:89
 #2  0x00007ffabe00b890 in malloc_printerr (action=<optimized out>,
 str=0x7ffabe0fdc27 "free(): invalid pointer", ptr=<optimized out>,
 ar_ptr=<optimized out>) at malloc.c:5008
 #3  0x00007ffabf470411 in vaapi_encode_config_attributes
 (avctx=0x55ef16408aa0) at src/libavcodec/vaapi_encode.c:1107
 #4  ff_vaapi_encode_init (avctx=0x55ef16408aa0) at
 src/libavcodec/vaapi_encode.c:1377
 #5  0x00007ffabf8d78f9 in avcodec_open2 (avctx=0x55ef16408aa0,
 codec=0x7ffac0304ce0 <ff_h264_vaapi_encoder>, options=0x55ef163a67f8) at
 src/libavcodec/utils.c:1020
 #6  0x000055ef142c8dde in init_output_stream (ost=<optimized out>,
 error=0x7fff49d94c20 "", error_len=1024) at src/ffmpeg.c:3438
 #7  0x000055ef142ca9c1 in reap_filters (flush=0) at src/ffmpeg.c:1443
 #8  0x000055ef142acd94 in transcode_step () at src/ffmpeg.c:4522
 #9  transcode () at src/ffmpeg.c:4566
 #10 main (argc=<optimized out>, argv=<optimized out>) at src/ffmpeg.c:477
 }}}
 The other two cases appear to work (albeit slowly, but that might be an
 r600 issue).

 MALLOC_CHECK_=3 :
 {{{
 *** Error in `ffmpeg': free(): invalid pointer: 0x0000560c61308b20 ***
 ======= Backtrace: =========
 /lib64/libc.so.6(+0x72a07)[0x7fea668a5a07]
 /lib64/libc.so.6(+0x78866)[0x7fea668ab866]
 /usr/lib64/libavcodec.so.57(+0xad411)[0x7fea67d10411]
 /usr/lib64/libavcodec.so.57(avcodec_open2+0x879)[0x7fea681778f9]
 ffmpeg(+0x28dde)[0x560c6094ddde]
 ffmpeg(+0x2a9c1)[0x560c6094f9c1]
 ffmpeg(+0xcd94)[0x560c60931d94]
 /lib64/libc.so.6(__libc_start_main+0xfc)[0x7fea668537cc]
 ffmpeg(+0xec69)[0x560c60933c69]
 ======= Memory map: ========
 }}}

--
Ticket URL: <https://trac.ffmpeg.org/ticket/6379#comment:4>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker