[FFmpeg-devel] rectification filter
Daniel Oberhoff
danieloberhoff at gmail.com
Sat Aug 9 22:41:51 CEST 2014
Am 08.08.2014 um 12:10 schrieb Michael Niedermayer <michaelni at gmx.at>:
> On Thu, Aug 07, 2014 at 12:15:17AM +0200, Daniel Oberhoff wrote:
>>
>> Am 06.08.2014 um 12:12 schrieb Clément Bœsch <u at pkh.me>:
>>
>>> On Sun, Aug 03, 2014 at 06:43:18PM +0200, Daniel Oberhoff wrote:
>>> [...]
>>>>>> +static av_cold int init(AVFilterContext *ctx)
>>>>>> +{
>>>>>> + return 0;
>>>>>> +}
>>>>>>
>>>>>
>>>>> please remove this one, there is no point in having code that does nothing.
>>>>>
>>>>>> +
>>>>>> +static av_cold void uninit(AVFilterContext *ctx)
>>>>>> +{
>>>>>> +}
>>>>>> +
>>>>>>
>>>>>
>>>>> ditto
>>>>>
>>>>
>>>> Done thanks!
>>>>
>>>
>>> Did you forget to attach the latest version of the patch or I missed it
>>> somewhere in the thread?
>>
>> Sorry, think that day I wanted to add something else before resubmit, and then lost the time.
>>
>> Here also with changes suggested by timothy. Note that the parametrization is now traditional
>> (i.e. k1 and k2 are zero-based, as for example in the opencv documentation about calibration).
>>
>> From b798b95eb2f9ec4043b2407970d351cc31dd9e6a Mon Sep 17 00:00:00 2001
>> From: Daniel Oberhoff <daniel at danieloberhoff.de>
>> Date: Mon, 28 Jul 2014 23:58:12 +0200
>> Subject: [PATCH] avfilter: ported lenscorrection filter from frei0r
>>
>> ---
>> Changelog | 2 +-
>> MAINTAINERS | 1 +
>> doc/filters.texi | 46 +++++++
>> libavfilter/Makefile | 1 +
>> libavfilter/allfilters.c | 1 +
>> libavfilter/version.h | 4 +-
>> libavfilter/vf_lenscorrection.c | 199 +++++++++++++++++++++++++++
>> tests/fate/filter-video.mak | 3 +
>> tests/ref/fate/filter-pixfmts-lenscorrection | 8 ++
>> 9 files changed, 262 insertions(+), 3 deletions(-)
>> create mode 100644 libavfilter/vf_lenscorrection.c
>> create mode 100644 tests/ref/fate/filter-pixfmts-lenscorrection
>>
>
> This is missing a "Signed-off-by" from you
So I what do I write there?
>> diff --git a/Changelog b/Changelog
>> index 25dd210..79393ce 100644
>> --- a/Changelog
>> +++ b/Changelog
>> @@ -3,7 +3,7 @@ releases are sorted from youngest to oldest.
>>
>> version <next>:
>> - Icecast protocol
>> -
>> +- ported lenscorrection filter from frei0r filter
>>
>> version 2.3:
>> - AC3 fixed-point decoding
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index c383d3f..6948e22 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -354,6 +354,7 @@ Filters:
>> vf_histogram.c Paul B Mahol
>> vf_hqx.c Clément Bœsch
>> vf_il.c Paul B Mahol
>> + vf_lenscorrection.c Daniel Oberhoff
>> vf_mergeplanes.c Paul B Mahol
>> vf_psnr.c Paul B Mahol
>> vf_scale.c Michael Niedermayer
>> diff --git a/doc/filters.texi b/doc/filters.texi
>> index 86feebc..c4888d3 100644
>> --- a/doc/filters.texi
>> +++ b/doc/filters.texi
>> @@ -5532,6 +5532,51 @@ kerndeint=map=1
>> @end example
>> @end itemize
>>
>> + at section lenscorrection
>> +
>> +Correct radial lens distortion
>> +
>> +This filter can be used to correct for radial distortion as can result from the use
>> +of wide angle lenses, and thereby re-rectify the image. To find the right parameters
>> +one can use tools available for example as part of opencv or simply trial-and-error.
>> +To use opencv use the calibration sample (under samples/cpp) from the opencv sources
>> +and extract the k1 and k2 coefficients from the resulting matrix.
>> +
>> +Note that effectively the same filter is available in the open-source tools Krita and
>> +Digikam from the KDE project.
>> +
>> +In contrast to the @ref{vignette} filter, which can also be used to compensate lens errors,
>> +this filter corrects the distortion of the image, whereas @ref{vignette} corrects the
>> +brightness distribution, so you may want to use both filters together in certain
>> +cases, though you will have to take care of ordering, i.e. whether vignetting should
>> +be applied before or after lens correction.
>> +
>> + at subsection Options
>> +
>> +The filter accepts the following options:
>> +
>> + at table @option
>> + at item cx
>> +Relative x-coordinate of the focal point of the image, and thereby the center of the
>> +distrortion. This value has a range [0,1] and is expressed as fractions of the image
>> +width.
>> + at item cy
>> +Relative y-coordinate of the focal point of the image, and thereby the center of the
>> +distrortion. This value has a range [0,1] and is expressed as fractions of the image
>> +height.
>> + at item k1
>> +Coefficient of the quadratic correction term. 0.5 means no correction.
>> + at item k2
>> +Coefficient of the double quadratic correction term. 0.5 means no correction.
>> + at end table
>> +
>> +The formula that generates the correction is:
>> +
>> + at var{r_src} = @var{r_tgt} * (1 + @var{k1} * (@var{r_tgt} / @var{r_0})^2 + @var{k2} * (@var{r_tgt} / @var{r_0})^4)
>> +
>> +where @var{r_0} is halve of the image diagonal and @var{r_src} and @var{r_tgt} are the
>> +distances from the focal point in the source and target images, respectively.
>> +
>> @anchor{lut3d}
>> @section lut3d
>>
>> @@ -8744,6 +8789,7 @@ For example, to vertically flip a video with @command{ffmpeg}:
>> ffmpeg -i in.avi -vf "vflip" out.avi
>> @end example
>>
>> + at anchor{vignette}
>> @section vignette
>>
>> Make or reverse a natural vignetting effect.
>> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
>> index 0f54381..e9c8456 100644
>> --- a/libavfilter/Makefile
>> +++ b/libavfilter/Makefile
>> @@ -138,6 +138,7 @@ OBJS-$(CONFIG_IL_FILTER) += vf_il.o
>> OBJS-$(CONFIG_INTERLACE_FILTER) += vf_interlace.o
>> OBJS-$(CONFIG_INTERLEAVE_FILTER) += f_interleave.o
>> OBJS-$(CONFIG_KERNDEINT_FILTER) += vf_kerndeint.o
>> +OBJS-$(CONFIG_LENSCORRECTION_FILTER) += vf_lenscorrection.o
>> OBJS-$(CONFIG_LUT3D_FILTER) += vf_lut3d.o
>> OBJS-$(CONFIG_LUT_FILTER) += vf_lut.o
>> OBJS-$(CONFIG_LUTRGB_FILTER) += vf_lut.o
>> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
>> index 1877557..b1d6ff5 100644
>> --- a/libavfilter/allfilters.c
>> +++ b/libavfilter/allfilters.c
>> @@ -156,6 +156,7 @@ void avfilter_register_all(void)
>> REGISTER_FILTER(INTERLACE, interlace, vf);
>> REGISTER_FILTER(INTERLEAVE, interleave, vf);
>> REGISTER_FILTER(KERNDEINT, kerndeint, vf);
>> + REGISTER_FILTER(LENSCORRECTION, lenscorrection, vf);
>> REGISTER_FILTER(LUT3D, lut3d, vf);
>> REGISTER_FILTER(LUT, lut, vf);
>> REGISTER_FILTER(LUTRGB, lutrgb, vf);
>> diff --git a/libavfilter/version.h b/libavfilter/version.h
>> index 1a43dc5..47bac78 100644
>> --- a/libavfilter/version.h
>> +++ b/libavfilter/version.h
>> @@ -30,8 +30,8 @@
>> #include "libavutil/version.h"
>>
>> #define LIBAVFILTER_VERSION_MAJOR 4
>> -#define LIBAVFILTER_VERSION_MINOR 11
>> -#define LIBAVFILTER_VERSION_MICRO 102
>> +#define LIBAVFILTER_VERSION_MINOR 12
>> +#define LIBAVFILTER_VERSION_MICRO 100
>>
>> #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \
>> LIBAVFILTER_VERSION_MINOR, \
>> diff --git a/libavfilter/vf_lenscorrection.c b/libavfilter/vf_lenscorrection.c
>> new file mode 100644
>> index 0000000..c2346db
>> --- /dev/null
>> +++ b/libavfilter/vf_lenscorrection.c
>> @@ -0,0 +1,199 @@
>> +/*
>> + * Copyright (C) 2007 Richard Spindler (author of frei0r plugin from which this was derived)
>> + * Copyright (C) 2014 Daniel Oberhoff
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
>> + */
>> +
>> +/**
>> + * @file
>> + * Lenscorrection filter, algorithm from the frei0r plugin with the same name
>> +*/
>> +#include <stdlib.h>
>> +#include <math.h>
>> +
>> +#include "libavutil/opt.h"
>> +#include "libavutil/intreadwrite.h"
>> +#include "libavutil/pixdesc.h"
>> +
>> +#include "avfilter.h"
>> +#include "internal.h"
>> +#include "video.h"
>> +
>> +typedef struct LenscorrectionCtx {
>> + const AVClass *av_class;
>> + unsigned int width;
>> + unsigned int height;
>> + int hsub, vsub;
>> + int nb_planes;
>> + double cx, cy, k1, k2;
>> +} LenscorrectionCtx;
>> +
>> +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
>> +static const AVOption lenscorrection_options[] = {
>> + { "cx", "set relative center x", offsetof(LenscorrectionCtx, cx), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, .flags=FLAGS },
>> + { "cy", "set relative center y", offsetof(LenscorrectionCtx, cy), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, .flags=FLAGS },
>> + { "k1", "set quadratic distortion factor", offsetof(LenscorrectionCtx, k1), AV_OPT_TYPE_DOUBLE, {.dbl=0.0}, -1, 1, .flags=FLAGS },
>> + { "k2", "set double quadratic distortion factor", offsetof(LenscorrectionCtx, k2), AV_OPT_TYPE_DOUBLE, {.dbl=0.0}, -1, 1, .flags=FLAGS },
>> + { NULL }
>> +};
>> +
>> +AVFILTER_DEFINE_CLASS(lenscorrection);
>> +
>> +typedef struct ThreadData {
>> + AVFrame *in, *out;
>> + float w, h;
>> + int plane;
>> + float xcenter, ycenter;
>> + float k1, k2;
>> +} ThreadData;
>> +
>> +static int filter_slice(AVFilterContext *ctx, void *arg, int job, int nb_jobs)
>> +{
>> + ThreadData *td = (ThreadData*)arg;
>> + AVFrame *in = td->in;
>> + AVFrame *out = td->out;
>> +
>> + const float w = td->w, h = td->h;
>> + const float xcenter = td->xcenter;
>> + const float ycenter = td->ycenter;
>> + const float r2inv = 4.0 / (w * w + h * h);
>> + const float k1 = td->k1;
>> + const float k2 = td->k2;
>> + const int start = (h * job ) / nb_jobs;
>> + const int end = (h * (job+1)) / nb_jobs;
>> + const int plane = td->plane;
>> + const int inlinesize = in->linesize[plane];
>> + const int outlinesize = out->linesize[plane];
>> + const uint8_t *indata = in->data[plane];
>> + uint8_t *outrow = out->data[plane] + start * outlinesize;
>> + int i;
>> + for (i = start; i < end; i++, outrow += outlinesize) {
>> + const float off_y = i - ycenter;
>> + const float off_y2 = off_y * off_y;
>> + uint8_t *out = outrow;
>> + int j;
>> + for (j = 0; j < w; j++) {
>> + const float off_x = j - xcenter;
>> + const float r2 = (off_x * off_x + off_y2) * r2inv;
>> + const float radius_mult = 1.0f + r2 * k1 + r2 * r2 * k2;
>> + const int x = xcenter + radius_mult * off_x + 0.5f;
>> + const int y = ycenter + radius_mult * off_y + 0.5f;
>> + const char isvalid = x > 0 && x < w - 1 && y > 0 && y < h - 1;
>
>> + *out++ = isvalid ? indata[y * inlinesize + x] : 0;
>
> IMHO "nearest neighbour" resampling isnt a reasonable choice, if
> thats the only choice the user has.
So you want me to add other methods? I found nearest neighbor sufficient for our needs and using other methods seemed to slow the code down too much for us, but if you think the filter is unfit for ffmpeg otherwise I will add other methods. Up to what order would you say resampling options are required?
> also md5 based regression tests and extensive use of floats are
> unlikely to work together, different architectures/platforms or even
> compiler versions might give you different results
Sounds reasonable, so should I remove the tests then?
Best
Daniel
More information about the ffmpeg-devel
mailing list