[FFmpeg-devel] [PATCH] libavfilter image transformations

[Alexander Strange]

On Apr 20, 2010, at 5:03 PM, Daniel G. Taylor wrote:

> Hey,
> 
> Attached is a patch to add matrix transformation utility methods to libavfilter. These are useful for translation, rotation, scaling, etc of input pictures and can be easily used from any filter.
> 
> This code was originally part of my work on the deshake/stabilize filter and the work of Georg Martius from his transcode transform filter.
> 
> Example usage:
> 
> #include "transform.h"
> 
> ...
> double matrix[9];
> 
> avfilter_get_matrix(shift_x, shift_y, radians, scale, (double *) &matrix);
> avfilter_transform(src, dst, src_stride, dst_stride, width, height, &matrix, INTERPOLATE_BILINEAR, FILL_MIRROR);
> ...
> 
> The code supports creating simple matrices from translation, rotation, and scaling parameters, adding/subtracting matrices, multiplying a matrix by a scalar, and doing the actual picture transform. It supports several types of interpolation and edge fill methods.
> 
> This is useful for any filter that might want to transform video frames in any way in the future. I have at least one filter that will use this but is waiting on motion estimation stuff (deshake/stabilize video).

First, are these transforms separable? I'm not familiar with affine rotations to know how to split them into 1D.

Second, can they be done in fixed-point?

Third, can it be merged with swscale? Though that looks hard, filters should be using it for scaling and transformation if possible.

> +#define INTERPOLATE_METHOD(name) static uint8_t name(float x, float y, const uint8_t *src, int width, int height, int stride, uint8_t def)
> +
> +#define PIXEL(img, x, y, w, h, stride, def) ((x) < 0 || (y) < 0) ? (def) : (((x) >= (w) || (y) >= (h)) ? (def) : img[(x) + (y) * (stride)])

Break these lines after the macro name.

> +        return PIXEL(src, (int)round(x), (int)round(y), width, height, stride, def);

Should be roundf(), and then it should be lrintf(). Unless (int)(x + .5f) is faster.
(Of course, ignore all of this if you can do it in fixed-point)

> +/**
> + * Bilinear interpolation
> + */


This won't work for downscaling because it won't sample any more than 4 source pixels. Same for the other methods.

> +        return (uint8_t) s;

The cast doesn't do anything.

> +/**
> + * Biquadratic interpolation
> + */

If you're going to provide something better than linear, at least make it bicubic or slower than that. swscale has several good kernels.

> +        x_c = (int)ceilf(x);
> +        x_f = (int)floorf(x);
> +        y_c = (int)ceilf(y);
> +        y_f = (int)floorf(y);

These casts are unnecessary too.

> +void avfilter_get_matrix(double x_shift, double y_shift, double angle, double zoom, double *matrix) {
> +    matrix[0] = zoom * cos(angle);
> +    matrix[1] = -sin(angle);
> +    matrix[2] = x_shift;
> +    matrix[3] = -matrix[1];
> +    matrix[4] = matrix[0];
> +    matrix[5] = y_shift;
> +    matrix[6] = 0;
> +    matrix[7] = 0;
> +    matrix[8] = 1;
> +}

Use float instead of double unless it's really unstable.

Just use a loop for add/sub/mul.

> +void avfilter_transform(const uint8_t *src, uint8_t *dst, int src_stride, int dst_stride, int width, int height, const double *matrix, enum InterpolateMethod interpolate, enum FillMethod fill)

This line is too long.

> +                case FILL_EXTRUDE:

Call it CLAMP, maybe?

> +                    def = src[(int)((y_s < 0) ? -y_s : (y_s >= height) ? (height + height - y_s) : y_s) * src_stride + (int)((x_s < 0) ? -x_s : (x_s >= width) ? (width + width - x_s) : x_s)];

Split this into multiple statements so the lines are shorter.