[FFmpeg-trac] #6379(avcodec:new): vaapi_encode_check_config invalid free

[FFmpeg]

#6379: vaapi_encode_check_config invalid free
----------------------------------+-----------------------------------
             Reporter:  serafean  |                    Owner:
                 Type:  defect    |                   Status:  new
             Priority:  normal    |                Component:  avcodec
              Version:  3.2.4     |               Resolution:
             Keywords:            |               Blocked By:
             Blocking:            |  Reproduced by developer:  0
Analyzed by developer:  0         |
----------------------------------+-----------------------------------

Comment (by serafean):

 Ah, my bad, I forgot about the free behaviour.

 Crashes.
 with MALLOC_CHECK_=1 segfault :
 {{{
 #0  _int_malloc (av=av at entry=0x7f8429308aa0 <main_arena>,
 bytes=bytes at entry=9) at malloc.c:3414
 #1  0x00007f8428fe635b in malloc_check (sz=sz at entry=8,
 caller=caller at entry=0x0) at hooks.c:295
 #2  0x00007f8428fe6d86 in realloc_check (oldmem=0x0, bytes=8,
 caller=<optimized out>) at hooks.c:355
 #3  0x00007f8429a80c4f in av_frame_new_side_data
 (frame=frame at entry=0x563b5c49d7a0, type=AV_FRAME_DATA_MATRIXENCODING,
 size=4) at src/libavutil/frame.c:634
 #4  0x00007f8429a80f40 in frame_copy_props (dst=dst at entry=0x563b5c49d7a0,
 src=0x563b5c4af0e0, force_copy=force_copy at entry=1) at
 src/libavutil/frame.c:339
 #5  0x00007f8429a810a9 in av_frame_copy_props
 (dst=dst at entry=0x563b5c49d7a0, src=<optimized out>) at
 src/libavutil/frame.c:591
 #6  0x00007f842be92141 in ff_filter_frame_needs_framing
 (frame=0x563b5c4af0e0, link=0x563b5c36c6c0) at
 src/libavfilter/avfilter.c:1162
 #7  ff_filter_frame (link=0x563b5c36c6c0, frame=0x563b5c4af0e0) at
 src/libavfilter/avfilter.c:1230
 #8  0x00007f842be90d9a in ff_filter_frame_framed
 (link=link at entry=0x563b5c36c540, frame=0x563b5c4af0e0) at
 src/libavfilter/avfilter.c:1134
 #9  0x00007f842be920c0 in ff_filter_frame (link=0x563b5c36c540,
 frame=0x563b5c4af0e0) at src/libavfilter/avfilter.c:1232
 #10 0x00007f842be90d9a in ff_filter_frame_framed
 (link=link at entry=0x563b5c36c2c0, frame=0x563b5c4af0e0) at
 src/libavfilter/avfilter.c:1134
 #11 0x00007f842be920c0 in ff_filter_frame (link=link at entry=0x563b5c36c2c0,
 frame=0x563b5c4af0e0) at src/libavfilter/avfilter.c:1232
 #12 0x00007f842be9658f in request_frame (link=0x563b5c36c2c0) at
 src/libavfilter/buffersrc.c:450
 #13 0x00007f842be96244 in av_buffersrc_add_frame_internal
 (ctx=ctx at entry=0x563b5c36b840, frame=frame at entry=0x563b5c45db60,
 flags=flags at entry=4)
     at src/libavfilter/buffersrc.c:239
 #14 0x00007f842be967ed in av_buffersrc_add_frame_flags
 (ctx=0x563b5c36b840, frame=0x563b5c45db60, flags=4) at
 src/libavfilter/buffersrc.c:164
 #15 0x0000563b5c065a7d in decode_audio (got_output=0x7ffdb8e80c0c,
 pkt=0x7ffdb8e80c30, ist=0x563b5c342ee0) at src/ffmpeg.c:2164
 #16 process_input_packet (ist=<optimized out>, pkt=0x7ffdb8e80f00,
 no_eof=0) at src/ffmpeg.c:2466
 #17 0x0000563b5c046daa in process_input (file_index=<optimized out>) at
 src/ffmpeg.c:4245
 #18 transcode_step () at src/ffmpeg.c:4333
 #19 transcode () at src/ffmpeg.c:4387
 #20 main (argc=<optimized out>, argv=<optimized out>) at src/ffmpeg.c:4592
 }}}
 Without MALLOC_CHECK_, sometimes SIGABRT:
 {{{
 *** Error in `ffmpeg': corrupted double-linked list: 0x000055ad35f86170
 ***
 ======= Backtrace: =========
 /lib64/libc.so.6(+0x72a07)[0x7f784ac7da07]
 /lib64/libc.so.6(+0x78866)[0x7f784ac83866]
 /lib64/libc.so.6(+0x78c21)[0x7f784ac83c21]
 /lib64/libc.so.6(+0x7a5d2)[0x7f784ac855d2]
 /lib64/libc.so.6(__libc_malloc+0x63)[0x7f784ac876f3]
 /usr/lib64/va/drivers/r600_drv_video.so(+0x210ca)[0x7f783bf140ca]
 /usr/lib64/va/drivers/r600_drv_video.so(+0x22fa2)[0x7f783bf15fa2]
 /usr/lib64/libavutil.so.55(+0x2072e)[0x7f784b72372e]
 /usr/lib64/libavutil.so.55(+0x20a77)[0x7f784b723a77]
 /usr/lib64/libavutil.so.55(av_hwframe_transfer_data+0xb7)[0x7f784b722ec7]
 /usr/lib64/libavfilter.so.6(+0x10ef89)[0x7f784dba1f89]
 /usr/lib64/libavfilter.so.6(+0x9dd9a)[0x7f784db30d9a]
 /usr/lib64/libavfilter.so.6(+0x9f0c0)[0x7f784db320c0]
 /usr/lib64/libavfilter.so.6(+0x9dd9a)[0x7f784db30d9a]
 /usr/lib64/libavfilter.so.6(+0x9f0c0)[0x7f784db320c0]
 /usr/lib64/libavfilter.so.6(+0x13d466)[0x7f784dbd0466]
 /usr/lib64/libavfilter.so.6(+0x9dd9a)[0x7f784db30d9a]
 /usr/lib64/libavfilter.so.6(+0x9f0c0)[0x7f784db320c0]
 /usr/lib64/libavfilter.so.6(+0xa358f)[0x7f784db3658f]
 /usr/lib64/libavfilter.so.6(+0xa3244)[0x7f784db36244]
 /usr/lib64/libavfilter.so.6(av_buffersrc_add_frame_flags+0xb5)[0x7f784db367ed]
 ffmpeg(+0x2bd05)[0x55ad34c4cd05]
 ffmpeg(+0xcdaa)[0x55ad34c2ddaa]
 /lib64/libc.so.6(__libc_start_main+0xfc)[0x7f784ac2b7cc]
 ffmpeg(+0xea59)[0x55ad34c2fa59]
 }}}
 running it through Valgrind throws out SIGILL.

 SIGSEGV in malloc is the most common.
 MALLOC_CHECK_=2 is the only one which is deterministic, so I thought that
 could be it...
 I'll try with ffmpeg 3.3 tomorrow.
 Do you think it could be in the r600 driver?

--
Ticket URL: <https://trac.ffmpeg.org/ticket/6379#comment:2>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker