[FFmpeg-trac] #943(undetermined:closed): Segfault libaacplus

[FFmpeg]

#943: Segfault libaacplus
-------------------------------------+-------------------------------------
             Reporter:  burek        |                    Owner:
                 Type:  defect       |                   Status:  closed
             Priority:  normal       |                Component:
              Version:  unspecified  |  undetermined
             Keywords:               |               Resolution:  invalid
             Blocking:               |               Blocked By:
Analyzed by developer:  0            |  Reproduced by developer:  0
-------------------------------------+-------------------------------------

Comment (by rectalogic):

 libaacplus encoding started crashing for all samples for me after
 upgrading from Ubuntu Lucid to Oneiric. The bug is probably somewhere in
 libaacplus or libfftw3, but adding info here for when others hit this.

 Oneiric now builds libfftw3 with SSE/SSE2 enabled, Lucid did not.
 https://bugs.launchpad.net/ubuntu/+source/fftw3/+bug/602586

 I tried building my own libfftw3 with and without --enable-sse/--enable-
 sse2 and if I enable SSE then libaacplus crashes, leave it disabled and it
 works.

 Here's a stack trace with libftw3 symbols when SSE is enabled:
 {{{
 Starting program: /home/aw/Projects/encoder/foundation/codex-
 foundation/scratch/7.0.4-dirty-debug/codex-foundation_7.0.4-dirty-
 debug_amd64/debian/opt/motionbox/foundation/7.0.4-dirty-debug/bin/ffmpeg
 -i /tmp/test.avi -codec:a libaacplus -b:a 64k -ar 44100 -ac 2 -vn -y
 /tmp/out.mp4
 [Thread debugging using libthread_db enabled]

 Program received signal SIGSEGV, Segmentation fault.
 0x00007ffff177af68 in LDA (x=0x6634cc, ivs=2, aligned_like=0x6634cc)
     at ../../../simd-support/simd-sse2.h:110
 110          return *(const V *)x;
 (gdb) p x
 $1 = (const R *) 0x6634cc
 (gdb) p *x
 $2 = 0
 #0  0x00007ffff177af68 in LDA (x=0x6634cc, ivs=2, aligned_like=0x6634cc)
     at ../../../simd-support/simd-sse2.h:110
 #1  0x00007ffff177b3a7 in t3fv_16 (ri=0x6634cc, ii=0x6634d0, W=0x6e2940,
     rs=0x6e2520, mb=0, me=32, ms=2) at ../common/t3fv_16.c:268
 #2  0x00007ffff14fa311 in apply (ego_=0x6def10, rio=0x6634cc,
 iio=0x6634d0)
     at dftw-direct.c:53
 #3  0x00007ffff14f9906 in apply_dit (ego_=0x6de7d0, ri=0x6634cc,
 ii=0x6634d0,
     ro=0x6634cc, io=0x6634d0) at ct.c:44
 #4  0x00007ffff167a4e6 in fftwf_execute_dft (p=0x63bc60, in=0x6634cc,
     out=0x6634cc) at execute-dft.c:29
 #5  0x00007ffff28204e4 in CFFTN (ctx=<optimized out>,
     afftData=<optimized out>, len=<optimized out>, isign=<optimized out>)
     at cfftn.c:1390
 #6  0x00007ffff283801e in mdct (fftctx=<optimized out>, dctdata=0x6634cc,
     sineWindow=<optimized out>, n=1024, ld_n=10, trigData=<optimized out>)
     at transform.c:128
 #7  0x00007ffff283836b in Transform_Real (fftctx=0x653ed8,
     mdctDelayBuffer=0x638220, timeSignal=0x653ef8, chIncrement=2,
     realOut=0x6634cc, blockType=<optimized out>) at transform.c:406
 #8  0x00007ffff282dd58 in psyMain (fftctx=0x653ed8, timeInStride=2,
     elemInfo=0x64f950, timeSignal=0x653ef8, psyData=0x651218,
     tnsData=0x652c88, psyConfLong=0x650998, psyConfShort=0x650fd0,
     psyOutChannel=0x650358, psyOutElement=0x650260, pScratchTns=0x6674cc)
     at psy_main.c:299
 #9  0x00007ffff2819bac in AacEncEncode (aacEnc=0x64f930,
     timeSignal=<optimized out>, timeInStride=<optimized out>,
     ancBytes=0x65af84 "\337\250\025\320\200", numAncBytes=<optimized out>,
     outBytes=<optimized out>, numOutBytes=0x7fffffffc7bc) at aacenc.c:310
 #10 0x00007ffff2838dbb in aacplusEncEncode (hEncoder=0x64f900,
     inputBuffer=<optimized out>, samplesInput=<optimized out>,
     outputBuffer=0x731d30 "S\277\366E", bufferSize=<optimized out>)
     at aacplusenc.c:349
 #11 0x00007ffff6a5a517 in aacPlus_encode_frame (avctx=0x637520,
     frame=0x731d30 "S\277\366E", buf_size=64584, data=0x6e3150)
     at libavcodec/libaacplus.c:106
 #12 0x00007ffff6bc6565 in avcodec_encode_audio (avctx=0x637520,
     buf=0x731d30 "S\277\366E", buf_size=64584, samples=0x6e3150)
     at libavcodec/utils.c:728
 #13 0x0000000000407378 in do_audio_out (s=0x636d70, ost=0x6379d0,
     ist=0x62e900,
     buf=0x702f20
 "\332\331\331\331a\302a\302\002\317\002\317+\344*\344\313\332\314\332N\330N\330\332\335\334\335\266\337\265\337e\367d\367\036\374\035\374{\333{\333\245\341\244\341",
 size=4096) at ffmpeg.c:970
 #14 0x000000000040ab91 in output_packet (ist=0x62e900, ist_index=1,
     ost_table=0x6379d0, nb_ostreams=1, pkt=0x7fffffffcce0) at
 ffmpeg.c:1872
 #15 0x000000000040e158 in transcode (output_files=0x637ce0,
 nb_output_files=1,
     input_files=0x62e980, nb_input_files=1) at ffmpeg.c:2632
 #16 0x00000000004150e2 in main (argc=14, argv=0x7fffffffe398) at
 ffmpeg.c:4473
 }}}

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/943#comment:5>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker