[FFmpeg-trac] #10280(avfilter:new): Request to add an option to the perspective filter to show through underlying content vs. extending the outside pixels

[FFmpeg]

#10280: Request to add an option to the perspective filter to show through
underlying content vs. extending the outside pixels
-------------------------------------+-------------------------------------
             Reporter:  jbmcgx       |                     Type:
                                     |  enhancement
               Status:  new          |                 Priority:  normal
            Component:  avfilter     |                  Version:
                                     |  unspecified
             Keywords:  perspective  |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 The perspective filter appears to currently only have the option to extend
 the outside pixels instead of letting the area outside the desired
 coordinates show through.

 With this command to overlay an image on another image with a perspective
 transform, you can see that the image underneath gets covered up:

 {{{
 ffmpeg -i "https://imposium-dev-
 uploads.s3.amazonaws.com/0ea8ab969efa8643b569db67be198db2/24IbheKbe5xbu0iehdv5Eb2ddd67Lci0W5G9/8.jpg"
 -i "https://imposium-dev-
 uploads.s3.amazonaws.com/0ea8ab969efa8643b569db67be198db2/4eea4565Rcf2S21051c6G8I6C060C018V1mc.jpg"
 -filter_complex "[1:v]
 scale=width=1920:height=1080:out_color_matrix=bt709:flags=lanczos,format=rgba,perspective=x0=534.49:y0=430.43:x1=867.1:y1=396.78:x2=526.01:y2=674.9:x3=887.03:y3=641.32:sense=destination
 [sized0]; [0:v][sized0] overlay=format=yuv444:x=0:y=0,format=yuv420p"
 -qscale:v 2 -frames:v 1 -pix_fmt yuvj420p "no_pad_extended_pixels.jpg"
 }}}

 I've discovered that you can get around this issue by adding a transparent
 border using the pad filter. This way the transparent pixels are extended
 instead of the edges of the overlaid image and the underlying image is
 shown. Here's that same command updated with that:

 {{{
 ffmpeg -i "https://imposium-dev-
 uploads.s3.amazonaws.com/0ea8ab969efa8643b569db67be198db2/24IbheKbe5xbu0iehdv5Eb2ddd67Lci0W5G9/8.jpg"
 -i "https://imposium-dev-
 uploads.s3.amazonaws.com/0ea8ab969efa8643b569db67be198db2/4eea4565Rcf2S21051c6G8I6C060C018V1mc.jpg"
 -filter_complex "[1:v]
 scale=width=1920:height=1080:out_color_matrix=bt709:flags=lanczos,pad=iw+4:ih+4:2:2:#00000000,format=rgba,perspective=x0=540.13:y0=430.7:x1=872.14:y1=397.37:x2=531.38:y2=675.01:x3=891.74:y3=641.77:sense=destination
 [sized0]; [0:v][sized0] overlay=format=yuv444:x=0:y=0,format=yuv420p"
 -qscale:v 2 -frames:v 1 -pix_fmt yuvj420p
 "with_pad_showing_underlying.jpg"
 }}}

 This works, however, it throws off the coordinates and it's not easy to
 calculate the resized border size and adjust accordingly.

 It would greatly improve usability to add an option to set the outside
 pixels to a certain colour or even just something like outer_pixels set to
 either "extend" or "transparent".
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10280>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker