[FFmpeg-devel] [PATCH] lavfi: add histeq filter (WIP)

Stefano Sabatini stefasab at gmail.com
Mon Oct 15 12:40:32 CEST 2012


On date Monday 2012-10-15 05:41:11 +0200, Jérémy Tran encoded:
> This is a port of virtual dub's histogram equalization filter by
> Donald A. Graft.
> 

> I reworked stefano's previous port (that used YUV colorspace) to use RGB
> colorspace like the original filter but the output is not correct yet.

In which sense is "not correct"? That may help to spot the issue.

> I am posting this patch to get a fist review, maybe there's something that
> I obviously got wrong.
> Thanks.
> ---
>  configure                |   1 +
>  doc/filters.texi         |  43 ++++++++
>  libavfilter/Makefile     |   1 +
>  libavfilter/allfilters.c |   1 +
>  libavfilter/vf_histeq.c  | 282 +++++++++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 328 insertions(+)
>  create mode 100644 libavfilter/vf_histeq.c
> 
> diff --git a/configure b/configure
> index 5aa891b..a72987c 100755
> --- a/configure
> +++ b/configure
> @@ -1916,6 +1916,7 @@ frei0r_filter_deps="frei0r dlopen"
>  frei0r_filter_extralibs='$ldl'
>  frei0r_src_filter_deps="frei0r dlopen"
>  frei0r_src_filter_extralibs='$ldl'
> +histeq_filter_deps="gpl"
>  hqdn3d_filter_deps="gpl"
>  movie_filter_deps="avcodec avformat"
>  mp_filter_deps="gpl avcodec swscale postproc inline_asm"
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 725c7b5..6bb293e 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -2285,6 +2285,49 @@ For example to horizontally flip the input video with @command{ffmpeg}:
>  ffmpeg -i in.avi -vf "hflip" out.avi
>  @end example
>  
> + at section histeq

> +From the original author's description:

Skip this sentence, since we may want to reword/rephrase it, breaking
the quote.

> +This filter applies a global color histogram equalization on a per-frame basis.
> +It can be used to correct video that has a compressed range of pixel intensities.
> +The filter redistributes the pixel intensities to equalize their distribution
> +across the intensity range. It may be viewed as an "automatically adjusting
> +contrast filter". This filter is useful only for correcting degraded or poorly
> +captured source video. It should not be applied willy-nilly, as it leads to a
> +modification of image content.
> +

> +The filter takes these optional parameters separated by ':':

av_opt_set_from_string() and named options, now that we have this luxury.

> + at var{strength}:@var{intensity}:@var{antibanding}
> +
> + at var{strength} must be an integer and determines the amount of equalization to
> +be applied. As the strength is reduced, the distribution of pixel intensities
> +more-and-more approaches that of the input frame.
> +It defaults to 200.
> +

> + at var{intensity} must be an integer and sets the maximum intensity that can
> +generated and scales the output values appropriately. The strength should be
> +set as desired and then the intensity can be limited if needed to avoid
> +washing-out.
> +It defaults to 210.

This doesn't tell the range of the value. I wonder if we should rather
adopt a more generic scale (float value in the range 0-1).

> +
> + at var{antibanding} must be 'none', 'weak' or 'strong' and randomly varies the
> +luminance of output pixels by a small amount to avoid banding of the histogram.
> +It defaults to 'none'.
> +
> +Some examples follow:
> + at itemize
> + at item
> +Use default values
> + at example
> +histeq
> + at end example
> +
> + at item
> +Use defaults strength and intensity values and enable strong antibanding
> + at example
> +histeq=200:210:strong
> + at end example
> + at end itemize
> +
>  @section hqdn3d
>  
>  High precision/quality 3d denoise filter. This filter aims to reduce
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 3618f10..bf82266 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -106,6 +106,7 @@ OBJS-$(CONFIG_FPS_FILTER)                    += vf_fps.o
>  OBJS-$(CONFIG_FREI0R_FILTER)                 += vf_frei0r.o
>  OBJS-$(CONFIG_GRADFUN_FILTER)                += vf_gradfun.o
>  OBJS-$(CONFIG_HFLIP_FILTER)                  += vf_hflip.o
> +OBJS-$(CONFIG_HISTEQ_FILTER)                 += vf_histeq.o
>  OBJS-$(CONFIG_HQDN3D_FILTER)                 += vf_hqdn3d.o
>  OBJS-$(CONFIG_HUE_FILTER)                    += vf_hue.o
>  OBJS-$(CONFIG_IDET_FILTER)                   += vf_idet.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 348f369..2d7c95f 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -98,6 +98,7 @@ void avfilter_register_all(void)
>      REGISTER_FILTER (FREI0R,      frei0r,      vf);
>      REGISTER_FILTER (GRADFUN,     gradfun,     vf);
>      REGISTER_FILTER (HFLIP,       hflip,       vf);
> +    REGISTER_FILTER (HISTEQ,      histeq,      vf);
>      REGISTER_FILTER (HQDN3D,      hqdn3d,      vf);
>      REGISTER_FILTER (HUE,         hue,         vf);
>      REGISTER_FILTER (IDET,        idet,        vf);
> diff --git a/libavfilter/vf_histeq.c b/libavfilter/vf_histeq.c
> new file mode 100644
> index 0000000..5eeeb3b
> --- /dev/null
> +++ b/libavfilter/vf_histeq.c
> @@ -0,0 +1,282 @@
> +/*
> + * Copyright (c) 2010 Stefano Sabatini
> + * Copyright (c) 2001 Donald A. Graft

Add your name here, feel free to discard mine if you did enough
changes to my original port (so we avoid another copyright soup).

> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License along
> + * with FFmpeg; if not, write to the Free Software Foundation, Inc.,
> + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> + */
> +
> +/**
> + * @file
> + * Histogram equalization filter, based on the VirtualDub filter by
> + * Donald A. Graft  <neuron2 AT home DOT com>.
> + * Implements global automatic contrast adjustment by means of
> + * histogram equalization.
> + */

Note, did you try to contact the author about the licensing? IIRC I
tried during the port, and I failed to get a reply.

> +
> +/* #define DEBUG */
> +
> +#include "libavutil/common.h"
> +#include "libavutil/pixdesc.h"
> +
> +#include "avfilter.h"
> +#include "formats.h"
> +#include "video.h"
> +
> +#define NONE   0
> +#define WEAK   1
> +#define STRONG 2
> +
> +#define R 0
> +#define G 1
> +#define B 2
> +#define A 3
> +
> +typedef struct {
> +    int strength;
> +    int intensity;
> +    int antibanding;
> +    int in_histogram [256];        ///< input histogram
> +    int out_histogram[256];        ///< output histogram
> +    int LUT[256];                  ///< lookup table derived from histogram[]

> +    unsigned int jran, ia, ic, im;

doxy

> +    int vsub, hsub;
> +    int rgba_map[4];               ///< components position
> +    int bpp;                       ///< bytes per pixel
> +} HisteqContext;
> +
> +static av_cold int init(AVFilterContext *ctx, const char *args)
> +{
> +    HisteqContext *histeq = ctx->priv;
> +    char antibanding_str[128] = "none";
> +
> +    histeq->strength    = 200;
> +    histeq->intensity   = 210;

> +    histeq->ia          = 4096;
> +    histeq->ic          = 150889;
> +    histeq->im          = 714025;

magic?

> +
> +    if (args)
> +        sscanf(args, "%d:%d:%s", &histeq->strength, &histeq->intensity, antibanding_str);

av_opt_set_from_string()

> +
> +    if      (!strcmp(antibanding_str, "none"  )) histeq->antibanding = NONE;
> +    else if (!strcmp(antibanding_str, "weak"  )) histeq->antibanding = WEAK;
> +    else if (!strcmp(antibanding_str, "strong")) histeq->antibanding = STRONG;
> +    else {
> +        av_log(ctx, AV_LOG_ERROR, "Unknown value for antibanding '%s'.\n", antibanding_str);
> +        return AVERROR(EINVAL);
> +    }
> +
> +    av_log(ctx, AV_LOG_INFO, "strength:%d intensity:%d antibanding:%s\n",
> +           histeq->strength, histeq->intensity, antibanding_str);
> +
> +    return 0;
> +}
> +
> +static int query_formats(AVFilterContext *ctx)
> +{
> +    static const enum PixelFormat pix_fmts[] = {
> +        PIX_FMT_ARGB,         PIX_FMT_RGBA,
> +        PIX_FMT_ABGR,         PIX_FMT_BGRA,
> +        PIX_FMT_NONE
> +    };
> +

> +    ff_set_common_formats(ctx, ff_make_format_list(pix_fmts));

Note: we should make this return a ret value.

> +
> +    return 0;
> +}
> +
> +static int config_input(AVFilterLink *inlink)
> +{
> +    AVFilterContext *ctx = inlink->dst;
> +    HisteqContext *histeq = ctx->priv;
> +    const AVPixFmtDescriptor *pix_desc = &av_pix_fmt_descriptors[inlink->format];
> +
> +    histeq->hsub = pix_desc->log2_chroma_w;
> +    histeq->vsub = pix_desc->log2_chroma_h;
> +    histeq->bpp = av_get_bits_per_pixel(pix_desc) / 8;
> +
> +    switch (inlink->format) {
> +        case PIX_FMT_ARGB:
> +            histeq->rgba_map[R] = 2;
> +            histeq->rgba_map[G] = 1;
> +            histeq->rgba_map[B] = 0;
> +            histeq->rgba_map[A] = 3;
> +            break;
> +        case PIX_FMT_RGBA:
> +            histeq->rgba_map[R] = 3;
> +            histeq->rgba_map[G] = 2;
> +            histeq->rgba_map[B] = 1;
> +            histeq->rgba_map[A] = 0;
> +            break;
> +        case PIX_FMT_ABGR:
> +            histeq->rgba_map[R] = 0;
> +            histeq->rgba_map[G] = 1;
> +            histeq->rgba_map[B] = 2;
> +            histeq->rgba_map[A] = 3;
> +            break;
> +        case PIX_FMT_BGRA:
> +            histeq->rgba_map[R] = 1;
> +            histeq->rgba_map[G] = 2;
> +            histeq->rgba_map[B] = 3;
> +            histeq->rgba_map[A] = 0;
> +            break;
> +        default:
> +            break;
> +    }
> +
> +    return 0;
> +}

libavfilter/drawutils.h:ff_fill_rgba_map

> +
> +#define COMPUTE_RGB_VALUES(r, g, b, src, map) do { \
> +    r = src[x + map[R]];                           \
> +    g = src[x + map[G]];                           \
> +    b = src[x + map[B]];                           \
> +} while (0)
> +
> +static int end_frame(AVFilterLink *inlink)
> +{
> +    AVFilterContext *ctx = inlink->dst;
> +    HisteqContext *histeq = ctx->priv;
> +    AVFilterBufferRef *inpic = inlink->cur_buf;
> +    AVFilterLink *outlink = ctx->outputs[0];
> +    AVFilterBufferRef *outpic = outlink->out_buf;
> +    int x, y, i, luthi, lutlo, lut, luma, oluma, m;
> +    unsigned int r, g, b;
> +    uint8_t *src, *dst;
> +

> +    /* Seed random generator for antibanding. */
> +    histeq->jran = 739187UL;
> +
> +    /* Calculate and store the luminance and calculate the global histogram
> +       based on the luminance. */
> +    memset(histeq->in_histogram, 0, sizeof(histeq->in_histogram));
> +    src = inpic->data[0];
> +    dst = outpic->data[0];
> +    for (y = 0; y < inlink->h; y++) {
> +        for (x = 0; x < inlink->w * histeq->bpp; x += histeq->bpp) {
> +            COMPUTE_RGB_VALUES(r, g, b, src, histeq->rgba_map);
> +            luma = (55 * r + 182 * g + 19 * b) >> 8;
> +            dst[x + histeq->rgba_map[A]] = luma;
> +            histeq->in_histogram[luma]++;
> +        }
> +        src += inpic->linesize[0];
> +        dst += outpic->linesize[0];
> +    }
> +#ifdef DEBUG
> +    for (x = 0; x < 256; x++)
> +        av_log(ctx, AV_LOG_DEBUG, "in[%d]: %u\n", x, histeq->in_histogram[x]);
> +#endif
> +
> +    /* Calculate the lookup table. */
> +    histeq->LUT[0] = histeq->in_histogram[0];
> +    /* Accumulate */
> +    for (x = 1; x < 256; x++)
> +        histeq->LUT[x] = histeq->LUT[x-1] + histeq->in_histogram[x];
> +
> +    /* Normalize. */
> +    for (x = 0; x < 256; x++)
> +        histeq->LUT[x] = (histeq->LUT[x] * histeq->intensity) / (inlink->h * inlink->w);
> +
> +    /* Adjust the LUT based on the selected strength. This is an alpha
> +       mix of the calculated LUT and a linear LUT with gain 1. */
> +    for (x = 0; x < 256; x++)
> +        histeq->LUT[x] = (histeq->strength * histeq->LUT[x]) / 255 +
> +                         ((255 - histeq->strength) * x)      / 255;
> +
> +    /* Output the equalized frame. */
> +    memset(histeq->out_histogram, 0, sizeof(histeq->out_histogram));
> +
> +    src = inpic->data[0];
> +    dst = outpic->data[0];
> +    for (y = 0; y < inlink->h; y++) {
> +        for (x = 0; x < inlink->w * histeq->bpp; x += histeq->bpp) {
> +            luma = dst[x + histeq->rgba_map[A]];
> +            if (luma == 0) {
> +                for (i = 0; i < histeq->bpp; ++i)
> +                    dst[x + i] = 0;
> +                histeq->out_histogram[0]++;
> +            } else {
> +                lut = histeq->LUT[luma];
> +                if (histeq->antibanding != NONE) {
> +                    if (luma > 0) {
> +                        lutlo = histeq->antibanding == WEAK ?
> +                                (histeq->LUT[luma] + histeq->LUT[luma - 1]) / 2 :
> +                                 histeq->LUT[luma - 1];
> +                    } else
> +                        lutlo = lut;
> +
> +                    if (luma < 255) {
> +                        luthi = (histeq->antibanding == WEAK) ?
> +                            (histeq->LUT[luma] + histeq->LUT[luma + 1]) / 2 :
> +                             histeq->LUT[luma + 1];
> +                    } else
> +                        luthi = lut;
> +
> +                    if (lutlo != luthi) {
> +                        histeq->jran = (histeq->jran * histeq->ia + histeq->ic) % histeq->im;
> +                        lut = lutlo + ((luthi - lutlo + 1) * histeq->jran) / histeq->im;
> +                    }
> +                }
> +
> +                COMPUTE_RGB_VALUES(r, g, b, src, histeq->rgba_map);
> +                if (((m = FFMAX(r, FFMAX(g, b))) * lut) / luma > 255) {
> +                    r = (r * 255) / m;
> +                    g = (g * 255) / m;
> +                    b = (b * 255) / m;
> +                } else {
> +                    r = (r * lut) / luma;
> +                    g = (g * lut) / luma;
> +                    b = (b * lut) / luma;
> +                }
> +                dst[x + histeq->rgba_map[R]] = r;
> +                dst[x + histeq->rgba_map[G]] = g;
> +                dst[x + histeq->rgba_map[B]] = b;
> +                oluma = (55 * r + 182 * g + 19 * b) >> 8;
> +                histeq->out_histogram[oluma]++;
> +            }
> +        }
> +        src += inpic->linesize[0];
> +        dst += outpic->linesize[0];
> +    }
> +#ifdef DEBUG
> +    for (x = 0; x < 256; x++)
> +        av_log(ctx, AV_LOG_DEBUG, "out[%d]: %u\n", x, histeq->out_histogram[x]);
> +#endif
> +
> +    return ff_end_frame(outlink);
> +}
> +
> +AVFilter avfilter_vf_histeq = {
> +    .name        = "histeq",
> +    .description = NULL_IF_CONFIG_SMALL("Perform global color histogram equalization."),
> +
> +    .priv_size = sizeof(HisteqContext),
> +    .init      = init,
> +    .query_formats = query_formats,
> +
> +    .inputs    = (AVFilterPad[]) {{ .name             = "default",

const AVFilterPad

> +                                    .type             = AVMEDIA_TYPE_VIDEO,
> +                                    .config_props     = config_input,
> +                                    .end_frame        = end_frame,
> +                                    .min_perms        = AV_PERM_WRITE | AV_PERM_READ,
> +                                    .rej_perms        = AV_PERM_PRESERVE },
> +                                  { .name = NULL}},
> +
> +    .outputs   = (AVFilterPad[]) {{ .name             = "default",

const AVFilterPad
-- 
FFmpeg = Faithful and Free Mere Peaceless Extroverse Guru


More information about the ffmpeg-devel mailing list