[FFmpeg-trac] #824(avcodec:new): Segfault while using h264 hardware acceleration

FFmpeg trac at avcodec.org
Fri Dec 23 18:47:05 CET 2011

#824: Segfault while using h264 hardware acceleration
             Reporter:  static-void  |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:  avcodec      |                  Version:  git-
             Keywords:               |  master
             Blocking:               |               Blocked By:
Analyzed by developer:  0            |  Reproduced by developer:  0
 I'm using FFmpeg integrated in mplayer (with VA-API pathes), and I'm
 getting a SIGSEGV while trying to watch h264 video with HW-acceleration.

 I've poked around with gdb and I think this is multithreading-related.
 After I added mplayer option "-lavdopts threads=1", which configures
 libavcodec, the problem has gone.

 === General info ===

 Crash happens inside a thread, in the start_frame function

     vactx->slice_param_size = sizeof(VASliceParameterBufferH264);
       ^-- vactx is NULL

 and vactx is AVCodecContext.hwaccel_context, which should be set to
 vaapi_context struct instance. But I've set a breakpoint and seen
 hwaccel_context being properly set at the init stage, although
 AVCodecContext pointer being not the same as the one in the place it
 crashes. I suppose it happens because AVCodecContext gets copied and its
 fields are replicated across worker threads (libavcodec/pthread.c), and I
 haven't seen hwaccel_context being copied in the code.

 Considering all above, I don't think it is needed to copy hwaccel_context,
 but rather to force threads=1 configuration on libavcodec when using HW
 acceleration. Several working threads would not improve anything in the
 case of HW accel, would they?

 === Debug info ===

 (gdb) run -vo vaapi:gl -va vaapi "<...>.mkv"
 Starting program: /usr/local/mplayer-vaapi-build/mplayer -vo vaapi:gl -va
 vaapi "<...>.mkv"
 [Thread debugging using libthread_db enabled]
 MPlayer SVN-r34365-4.6.1 (C) 2000-2011 MPlayer Team

 Playing <...>.mkv.
 libavformat file format detected.
 [matroska,webm @ 0xc798e0]max_analyze_duration 5000000 reached at 5019000
 [lavf] stream 0: video (h264), -vid 0
 [lavf] stream 1: audio (aac), -aid 0, -alang jpn
 [lavf] stream 2: subtitle (ass), -sid 0, -slang eng
 VIDEO:  [H264]  1280x720  0bpp  23.976 fps    0.0 kbps ( 0.0 kbyte/s)
 Load subtitles in <...>
 [vo_vaapi] Using OpenGL rendering
 libva: libva version 0.32.0
 Xlib:  extension "XFree86-DRI" missing on display ":0.0".
 libva: va_getDriverName() returns 0
 libva: Trying to open /usr/lib/dri/fglrx_drv_video.so
 libva: va_openDriver() returns 0
 Opening video filter: [ass auto=1]
 Couldn't open video filter 'ass'.
 ASS: cannot add video filter
 Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
 [New Thread 0x7fffe13b6700 (LWP 12803)]
 [New Thread 0x7fffe0bb5700 (LWP 12804)]
 Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
 Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
 AUDIO: 48000 Hz, 2 ch, s16le, 0.0 kbit/0.00% (ratio: 0->192000)
 Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4
 [AO SDL] Samplerate: 48000Hz Channels: Stereo Format s16le
 [New Thread 0x7fffdc3b3700 (LWP 12805)]
 [Thread 0x7fffdc3b3700 (LWP 12805) exited]
 [New Thread 0x7fffdc3b3700 (LWP 12806)]
 AO: [sdl] 48000Hz 2ch s16le (2 bytes per sample)
 Starting playback...
 Unsupported PixelFormat 61
 [VD_FFMPEG] Trying pixfmt=1.
 Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
 VO: [vaapi] 1280x720 => 1280x720 H.264 VA-API Acceleration
 [VD_FFMPEG] XVMC-accelerated MPEG-2.

 Program received signal SIGSEGV, Segmentation fault.
 [Switching to Thread 0x7fffe13b6700 (LWP 12803)]
 start_frame (avctx=0x1b098c0, buffer=0x0, size=0) at
 234         vactx->slice_param_size = sizeof(VASliceParameterBufferH264);

 (gdb) bt
 #0  start_frame (avctx=0x1b098c0, buffer=0x0, size=0) at
 #1  0x00000000008724bb in decode_nal_units (h=0x2504250, buf=0x1b1b780 "",
 buf_size=2752) at libavcodec/h264.c:3920
 #2  0x0000000000872c4a in decode_frame (avctx=0x1b098c0, data=0x1e53080,
 data_size=0x1e53200, avpkt=<optimized out>)
     at libavcodec/h264.c:4105
 #3  0x0000000000945818 in frame_worker_thread (arg=0x1e52f20) at
 #4  0x00007ffff6170efc in start_thread () from /lib/x86_64-linux-
 #5  0x00007ffff055389d in clone () from /lib/x86_64-linux-gnu/libc.so.6
 #6  0x0000000000000000 in ?? ()

 (gdb) disass /m $pc-32,+64
 Dump of assembler code from 0x9af3a7 to 0x9af3e7:
 225     {
    0x00000000009af3ad <start_frame+0>:  push   %r14
    0x00000000009af3af <start_frame+2>:  push   %r13
    0x00000000009af3b1 <start_frame+4>:  push   %r12
    0x00000000009af3b3 <start_frame+6>:  push   %rbp
    0x00000000009af3b4 <start_frame+7>:  push   %rbx
    0x00000000009af3b5 <start_frame+8>:  sub    $0x10,%rsp

 226         H264Context * const h = avctx->priv_data;
    0x00000000009af3b9 <start_frame+12>: mov    0x98(%rdi),%rbx

 227         MpegEncContext * const s = &h->s;
 228         struct vaapi_context * const vactx = avctx->hwaccel_context;
    0x00000000009af3c0 <start_frame+19>: mov    0x3e8(%rdi),%r12

 229         VAPictureParameterBufferH264 *pic_param;
 230         VAIQMatrixBufferH264 *iq_matrix;
 232         av_dlog(avctx, "start_frame()\n");
 234         vactx->slice_param_size = sizeof(VASliceParameterBufferH264);
 => 0x00000000009af3c7 <start_frame+26>: movl   $0x828,0x38(%r12)

 236         /* Fill in VAPictureParameterBufferH264. */
 237         pic_param = ff_vaapi_alloc_pic_param(vactx,
    0x00000000009af3d0 <start_frame+35>: mov    $0x170,%esi
    0x00000000009af3d5 <start_frame+40>: mov    %r12,%rdi
    0x00000000009af3d8 <start_frame+43>: callq  0xa9f079
    0x00000000009af3dd <start_frame+48>: mov    %rax,%rbp

 238         if (!pic_param)
    0x00000000009af3e0 <start_frame+51>: test   %rax,%rax
 ---Type <return> to continue, or q <return> to quit---q

 (gdb) info locals
 h = 0x2504250
 s = 0x2504250
 vactx = 0x0
 pic_param = <optimized out>
 iq_matrix = <optimized out>

 (gdb) info registers
 rax            0x10995a0        17405344
 rbx            0x2504250        38814288
 rcx            0x0      0
 rdx            0x0      0
 rsi            0x0      0
 rdi            0x1b098c0        28350656
 rbp            0x2504250        0x2504250
 rsp            0x7fffe13b5c90   0x7fffe13b5c90
 r8             0x1      1
 r9             0x0      0
 r10            0x0      0
 r11            0x2      2
 r12            0x0      0
 r13            0x0      0
 r14            0x0      0
 r15            0x415f   16735
 rip            0x9af3c7 0x9af3c7 <start_frame+26>
 eflags         0x10206  [ PF IF RF ]
 cs             0x33     51
 ss             0x2b     43
 ds             0x0      0
 es             0x0      0
 fs             0x0      0
 gs             0x0      0

Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/824>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker

More information about the FFmpeg-trac mailing list