[FFmpeg-trac] #9415(swscale:new): swscale/yuv2yuvX yasm port broke non-yasm builds

[FFmpeg]

#9415: swscale/yuv2yuvX yasm port broke non-yasm builds
-------------------------------------+-------------------------------------
             Reporter:  Ben Boeckel  |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:  swscale      |                  Version:
                                     |  unspecified
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 VTK's AVI writer started having a strange behavior when writing out AVI
 files where the result would have stripes which change colors "scan in"
 over and over. Attached is the resulting file with this behavior. The
 relevant FFmpeg code in VTK can be see here:

 https://gitlab.kitware.com/vtk/vtk/-/blob/master/IO/FFMPEG/vtkFFMPEGWriter.cxx

 Reverting the following commits resolves the problem when applied to 4.4,
 but not master (more debugging is needed there):

         c00567647e9002094255df755e18c719e75b3333 "swscale/x86/swscale: fix
 mix of inline and external function definitions"
         c2bf1dcace8d331e672b955f9cf5b59211749f00 "swscale/x86/swscale: fix
 compilation with old yasm"
         1371647fc381774d40b8beecf69604331262172d "checkasm/sw_scale: use
 av_free() instead of free()"
         d52ceed9fd3655113657d9b572fc2c83b7f913cc "tests/checkasm/sw_scale:
 use memset() to fill dither"
         ee18edb13a9ae3041df961dd5003c2055b5cab35 "checkasm/sw_scale:
 properly initialize src_pixer and filter_coeff buffers"
         6bc2058d00b119d265c9970eac213d2922d15129 "tests/checkasm/sw_scale:
 adds additional tests sizes for yux2yuvX"
         c23a5523b5c5bc63d206cd18a669875034e62ae5 "swscale/x86/swscale:
 Remove unused ASM constants"
         95aacf30e3803c57d91ff62975b375e394f61d49 "libswscale/x86/swscale:
 Only call ff_yuv2yuvX functions if the input size is > 0"
         d512ebbaedefc977d7628da4dc3ecfe4e52a066a "swscale/x86/yuv2yuvX:
 use the SPLATW helper macro"
         ebb48d85a07551ca82965bb66b88e50606683b4d "swscale/x86/yuv2yuvX:
 use movq to load 8 bytes in all non-AVX2 functions"
         1a555d3c604804dcedacd230d410cfc822da3f4c "swscale/x86/yuv2yuvX:
 use the movsxdifnidn helper macro"
         4aeedf4c2a8f35be667d5dd40c84bd27730ef1db "libswscale/x86/yuv2yuvX:
 Removes unrolling for mmx and mmxext"
         554c2bc7086f49ef5a6a989ad6bc4bc11807eb6f "swscale: move
 yuv2yuvX_sse3 to yasm, unrolls main loop"

 FFmpeg is configured with:

     --disable-x86asm --prefix=… --enable-shared --disable-static

 to test this behavior.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9415>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker