libavfilter/libmpcodecs/vf_remove_logo.c File Reference

Advanced blur-based logo removing filter. More...

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <inttypes.h>
#include "config.h"
#include "mp_msg.h"
#include "libvo/fastmemcpy.h"
#include "img_format.h"
#include "mp_image.h"
#include "vf.h"

Go to the source code of this file.

Data Structures

struct pgm_structure

Simple implementation of the PGM image format. More...

struct vf_priv_s

Stores persistant variables. More...

Defines

#define max(x, y)   ((x)>(y)?(x):(y))

Returns the larger of the two arguments.

#define min(x, y)   ((x)>(y)?(y):(x))

Returns the smaller of the two arguments.

#define test_filter(image, x, y)   ((unsigned char) (image->pixel[((y) * image->width) + (x)]))

Test if a pixel is part of the logo.

#define apply_mask_fudge_factor(x)   (((x) >> 2) + x)

Chooses a slightly larger mask size to improve performance.

#define REMOVE_LOGO_LOAD_PGM_ERROR_MESSAGE(message)   {mp_msg(MSGT_VFILTER, MSGL_ERR, message); return NULL;}

Functions

static void * safe_malloc (int size)

Mallocs memory and checks to make sure it succeeded.

static void calculate_bounding_rectangle (int *posx1, int *posy1, int *posx2, int *posy2, pgm_structure *filter)

Calculates the smallest rectangle that will encompass the logo region.

static void destroy_masks (vf_instance_t *vf)

Free mask memory.

static void initialize_masks (vf_instance_t *vf)

Set up our array of masks.

static void convert_mask_to_strength_mask (vf_instance_t *vf, pgm_structure *mask)

Pre-processes an image to give distance information.

static void get_blur (const vf_instance_t *const vf, unsigned int *const value_out, const pgm_structure *const logo_mask, const mp_image_t *const image, const int x, const int y, const int plane)

Our blurring function.

static void destroy_pgm (pgm_structure *to_be_destroyed)

Free a pgm_structure.

static void load_pgm_skip (FILE *f)

Helper function for load_pgm(.

static pgm_structure * load_pgm (const char *file_name)

Loads a raw pgm or ppm file into a newly created pgm_structure object.

static pgm_structure * generate_half_size_image (vf_instance_t *vf, pgm_structure *input_image)

Generates a scaled down image with half width, height, and intensity.

static unsigned int find_best (struct vf_instance *vf)

Checks if YV12 is supported by the next filter.

static int config (struct vf_instance *vf, int width, int height, int d_width, int d_height, unsigned int flags, unsigned int outfmt)

Configure the filter and call the next filter's config function.

static void convert_yv12 (const vf_instance_t *const vf, const char *const source, const int source_stride, const mp_image_t *const source_image, const int width, const int height, char *const destination, const int destination_stride, int is_image_direct, pgm_structure *filter, const int plane, const int logo_start_x, const int logo_start_y, const int logo_end_x, const int logo_end_y)

Removes the logo from a plane (either luma or chroma).

static int put_image (struct vf_instance *vf, mp_image_t *mpi, double pts)

Process a frame.

static int query_format (struct vf_instance *vf, unsigned int fmt)

Checks to see if the next filter accepts YV12 images.

static void uninit (vf_instance_t *vf)

Frees memory that our filter allocated.

static int vf_open (vf_instance_t *vf, char *args)

Initializes our filter.

Variables

const vf_info_t vf_info_remove_logo

Meta data about our filter.

Detailed Description

Advanced blur-based logo removing filter.

Hello and welcome. This code implements a filter to remove annoying TV logos and other annoying images placed onto a video stream. It works by filling in the pixels that comprise the logo with neighboring pixels. The transform is very loosely based on a gaussian blur, but it is different enough to merit its own paragraph later on. It is a major improvement on the old delogo filter as it both uses a better blurring algorithm and uses a bitmap to use an arbitrary and generally much tighter fitting shape than a rectangle.

The filter requires 1 argument and has no optional arguments. It requires a filter bitmap, which must be in PGM or PPM format. A sample invocation would be -vf remove_logo=/home/username/logo_bitmaps/xyz.pgm. Pixels with a value of zero are not part of the logo, and non-zero pixels are part of the logo. If you use white (255) for the logo and black (0) for the rest, you will be safe. For making the filter bitmap, I recommend taking a screen capture of a black frame with the logo visible, and then using The GIMP's threshold filter followed by the erode filter once or twice. If needed, little splotches can be fixed manually. Remember that if logo pixels are not covered, the filter quality will be much reduced. Marking too many pixels as part of the logo doesn't hurt as much, but it will increase the amount of blurring needed to cover over the image and will destroy more information than necessary. Additionally, this blur algorithm is O(n) = n^4, where n is the width and height of a hypothetical square logo, so extra pixels will slow things down on a large lo

The logo removal algorithm has two key points. The first is that it distinguishes between pixels in the logo and those not in the logo by using the passed-in bitmap. Pixels not in the logo are copied over directly without being modified and they also serve as source pixels for the logo fill-in. Pixels inside the logo have the mask applied.

At init-time the bitmap is reprocessed internally, and the distance to the nearest edge of the logo (Manhattan distance), along with a little extra to remove rough edges, is stored in each pixel. This is done using an in-place erosion algorithm, and incrementing each pixel that survives any given erosion. Once every pixel is eroded, the maximum value is recorded, and a set of masks from size 0 to this size are generaged. The masks are circular binary masks, where each pixel within a radius N (where N is the size of the mask) is a 1, and all other pixels are a 0. Although a gaussian mask would be more mathematically accurate, a binary mask works better in practice because we generally do not use the central pixels in the mask (because they are in the logo region), and thus a gaussian mask will cause too little blur and thus a very unstable image.

The mask is applied in a special way. Namely, only pixels in the mask that line up to pixels outside the logo are used. The dynamic mask size means that the mask is just big enough so that the edges touch pixels outside the logo, so the blurring is kept to a minimum and at least the first boundary condition is met (that the image function itself is continuous), even if the second boundary condition (that the derivative of the image function is continuous) is not met. A masking algorithm that does preserve the second boundary coundition (perhaps something based on a highly-modified bi-cubic algorithm) should offer even better results on paper, but the noise in a typical TV signal should make anything based on derivatives hopelessly noisy.

Definition in file vf_remove_logo.c.

Define Documentation

#define apply_mask_fudge_factor ( x ) (((x) >> 2) + x)

Chooses a slightly larger mask size to improve performance.

This function maps the absolute minimum mask size needed to the mask size we'll actually use. f(x) = x (the smallest that will work) will produce the sharpest results, but will be quite jittery. f(x) = 1.25x (what I'm using) is a good tradeoff in my opinion. This will calculate only at init-time, so you can put a long expression here without effecting performance.

Definition at line 118 of file vf_remove_logo.c.

Referenced by convert_mask_to_strength_mask(), and generate_half_size_image().

#define max	(	x,
		y		)	((x)>(y)?(x):(y))

Returns the larger of the two arguments.

Definition at line 100 of file vf_remove_logo.c.

Referenced by adx_encode(), block_metrics_mmx2(), calc_input_response(), categorize(), check_diff(), clean_index(), config_props(), dct_quantize_bfin(), dct_quantize_c(), dct_quantize_trellis_c(), decode_rice(), decode_slice_header(), dering_altivec(), dering_TMPL(), diff_plane(), direct_search(), do_a_deblock_C(), dts_probe(), dv_init_enc_block(), encode_cblk(), ff_ac3_bit_alloc_calc_psd(), ff_convert_matrix(), ff_nelly_get_sample_bits(), generate_half_size_image(), generate_new_codebooks(), get_blur(), get_new_centroids(), h264_deblock_q1(), isVertMinMaxOk_C(), main(), motion_search(), parse(), postProcess_TMPL(), put_image(), qdm2_decode_fft_packets(), sws_printVec2(), and x8_loop_filter().

#define min	(	x,
		y		)	((x)>(y)?(y):(x))

Returns the smaller of the two arguments.

Definition at line 102 of file vf_remove_logo.c.

Referenced by adx_encode(), array_min_int16(), av_set_number2(), avfilter_poll_frame(), calc_input_response(), categorize(), config_props(), dering_altivec(), dering_TMPL(), direct_search(), dnxhd_encode_rdo(), do_a_deblock_C(), generate_half_size_image(), get_blur(), get_new_centroids(), get_pts(), h264_deblock_q1(), initFilter(), isVertMinMaxOk_C(), pixel_color7_slow(), pred_intra_mode(), qdm2_decode_fft_packets(), sws_printVec2(), and x8_loop_filter().

#define REMOVE_LOGO_LOAD_PGM_ERROR_MESSAGE ( message ) {mp_msg(MSGT_VFILTER, MSGL_ERR, message); return NULL;}

Definition at line 513 of file vf_remove_logo.c.

Referenced by load_pgm().

#define test_filter	(	image,
		x,
		y	)	((unsigned char) (image->pixel[((y) * image->width) + (x)]))

Test if a pixel is part of the logo.

Definition at line 107 of file vf_remove_logo.c.

Referenced by calculate_bounding_rectangle(), and get_blur().

Function Documentation

static void calculate_bounding_rectangle	(	int *	posx1,
		int *	posy1,
		int *	posx2,
		int *	posy2,
		pgm_structure *	filter
	)			`[static]`

Calculates the smallest rectangle that will encompass the logo region.

Parameters:

filter

This image contains the logo around which the rectangle will will be fitted.

The bounding rectangle is calculated by testing successive lines (from the four sides of the rectangle) until no more can be removed without removing logo pixels. The results are returned by reference to posx1, posy1, posx2, and posy2.

Definition at line 193 of file vf_remove_logo.c.

Referenced by vf_open().

static int config	(	struct vf_instance *	vf,
		int	width,
		int	height,
		int	d_width,
		int	d_height,
		unsigned int	flags,
		unsigned int	outfmt
	)			`[static]`

Configure the filter and call the next filter's config function.

Definition at line 686 of file vf_remove_logo.c.

static void convert_mask_to_strength_mask	(	vf_instance_t *	vf,
		pgm_structure *	mask
	)			`[static]`

Pre-processes an image to give distance information.

Parameters:

	vf	Data structure that holds persistant information. All it is used for in this function is to store the calculated max_mask_size variable.
	mask	This image will be converted from a greyscale image into a distance image.

This function takes a greyscale image (pgm_structure * mask) and converts it in place into a distance image. A distance image is zero for pixels ourside of the logo and is the manhattan distance (|dx| + |dy|) for pixels inside of the logo. This will overestimate the distance, but that is safe, and is far easier to implement than a proper pythagorean distance since I'm using a modified erosion algorithm to compute the distances.

Definition at line 347 of file vf_remove_logo.c.

Referenced by vf_open().

static void convert_yv12	(	const vf_instance_t *const	vf,
		const char *const	source,
		const int	source_stride,
		const mp_image_t *const	source_image,
		const int	width,
		const int	height,
		char *const	destination,
		const int	destination_stride,
		int	is_image_direct,
		pgm_structure *	filter,
		const int	plane,
		const int	logo_start_x,
		const int	logo_start_y,
		const int	logo_end_x,
		const int	logo_end_y
	)			`[static]`

Removes the logo from a plane (either luma or chroma).

Parameters:

	vf	Not needed by this function, but needed by the blur function.
	source	The image to have it's logo removed.
	destination	Where the output image will be stored.
	source_stride	How far apart (in memory) two consecutive lines are.
	destination	Same as source_stride, but for the destination image.
	width	Width of the image. This is the same for source and destination.
	height	Height of the image. This is the same for source and destination.
	is_image_direct	If the image is direct, then source and destination are the same and we can save a lot of time by not copying pixels that haven't changed.
	filter	The image that stores the distance to the edge of the logo for each pixel.
	logo_start_x	Smallest x-coordinate that contains at least 1 logo pixel.
	logo_start_y	Smallest y-coordinate that contains at least 1 logo pixel.
	logo_end_x	Largest x-coordinate that contains at least 1 logo pixel.
	logo_end_y	Largest y-coordinate that contains at least 1 logo pixel.

This function processes an entire plane. Pixels outside of the logo are copied to the output without change, and pixels inside the logo have the de-blurring function applied.

Definition at line 718 of file vf_remove_logo.c.

Referenced by put_image().

static void destroy_masks ( vf_instance_t * vf ) [static]

Free mask memory.

Parameters:

vf

Data structure which stores our persistant data, and is to be freed.

We call this function when our filter is done. It will free the memory allocated to the masks and leave the variables in a safe state.

Definition at line 262 of file vf_remove_logo.c.

Referenced by uninit().

static void destroy_pgm ( pgm_structure * to_be_destroyed ) [static]

Free a pgm_structure.

Undoes load_pgm(...).

Definition at line 484 of file vf_remove_logo.c.

Referenced by uninit().

static unsigned int find_best ( struct vf_instance * vf ) [static]

Checks if YV12 is supported by the next filter.

Definition at line 673 of file vf_remove_logo.c.

static pgm_structure* generate_half_size_image	(	vf_instance_t *	vf,
		pgm_structure *	input_image
	)			`[static]`

Generates a scaled down image with half width, height, and intensity.

Parameters:

	vf	Our struct for persistant data. In this case, it is used to update mask_max_size with the larger of the old or new value.
	input_image	The image from which the new half-sized one will be based.

Returns:: The newly allocated and shrunken image.

This function not only scales down an image, but halves the value in each pixel too. The purpose of this is to produce a chroma filter image out of a luma filter image. The pixel values store the distance to the edge of the logo and halving the dimensions halves the distance. This function rounds up, because a downwards rounding error could cause the filter to fail, but an upwards rounding error will only cause a minor amount of excess blur in the chroma planes.

Definition at line 592 of file vf_remove_logo.c.

Referenced by vf_open().

static void get_blur	(	const vf_instance_t *const	vf,
		unsigned int *const	value_out,
		const pgm_structure *const	logo_mask,
		const mp_image_t *const	image,
		const int	x,
		const int	y,
		const int	plane
	)			`[static]`

Our blurring function.

Parameters:

	vf	Stores persistant data. In this function we are interested in the array of masks.
	value_out	The properly blurred and delogoed pixel is outputted here.
	logo_mask	Tells us which pixels are in the logo and which aren't.
	image	The image that is having its logo removed.
	x	x-coordinate of the pixel to blur.
	y	y-coordinate of the pixel to blur.
	plane	0 = luma, 1 = blue chroma, 2 = red chroma (YUV).

This function is the core of the filter. It takes a pixel that is inside the logo and blurs it. It does so by finding the average of all the pixels within the mask and outside of the logo.

Definition at line 428 of file vf_remove_logo.c.

Referenced by convert_yv12().

static void initialize_masks ( vf_instance_t * vf ) [static]

Set up our array of masks.

Parameters:

vf

Where our filter stores persistance data, like these masks.

This creates an array of progressively larger masks and calculates their values. The values will not change during program execution once this function is done.

Definition at line 299 of file vf_remove_logo.c.

Referenced by vf_open().

static pgm_structure* load_pgm ( const char * file_name ) [static]

Loads a raw pgm or ppm file into a newly created pgm_structure object.

Parameters:

file_name

The name of the file to be loaded. So long as the file is a valid pgm or ppm file, it will load correctly, even if the extension is missing or invalid.

Returns:: A pointer to the newly created pgm_structure object. Don't forget to call destroy_pgm(...) when you're done with this. If an error occurs, NULL is returned.

Can load either raw pgm (P5) or raw ppm (P6) image files as a binary image. While a pgm file will be loaded normally (greyscale), the only thing that is guaranteed with ppm is that all zero (R = 0, G = 0, B = 0) pixels will remain zero, and non-zero pixels will remain non-zero.

Definition at line 531 of file vf_remove_logo.c.

Referenced by vf_open().

static void load_pgm_skip ( FILE * f ) [static]

Helper function for load_pgm(.

..) to skip whitespace.

Definition at line 501 of file vf_remove_logo.c.

Referenced by load_pgm().

static int put_image	(	struct vf_instance *	vf,
		mp_image_t *	mpi,
		double	pts
	)			`[static]`

Process a frame.

Parameters:

	mpi	The image sent to use by the previous filter.
	dmpi	Where we will store the processed output image.
	vf	This is how the filter gets access to it's persistant data.

Returns:: The return code of the next filter, or 0 on failure/error.

This function processes an entire frame. The frame is sent by the previous filter, has the logo removed by the filter, and is then sent to the next filter.

Definition at line 767 of file vf_remove_logo.c.

static int query_format	(	struct vf_instance *	vf,
		unsigned int	fmt
	)			`[static]`

Checks to see if the next filter accepts YV12 images.

Definition at line 814 of file vf_remove_logo.c.

static void* safe_malloc ( int size ) [static]

Mallocs memory and checks to make sure it succeeded.

Parameters:

size

How many bytes to allocate.

Returns:: A pointer to the freshly allocated memory block, or NULL on failutre.

Mallocs memory, and checks to make sure it was successfully allocated. Because of how MPlayer works, it cannot safely halt execution, but at least the user will get an error message before the segfault happens.

Definition at line 173 of file vf_remove_logo.c.

Referenced by generate_half_size_image(), initialize_masks(), load_pgm(), and vf_open().

static void uninit ( vf_instance_t * vf ) [static]

Frees memory that our filter allocated.

This is called at exit-time.

Definition at line 827 of file vf_remove_logo.c.

static int vf_open	(	vf_instance_t *	vf,
		char *	args
	)			`[static]`

Initializes our filter.

Parameters:

args

The arguments passed in from the command line go here. This filter expects only a single argument telling it where the PGM or PPM file that describes the logo region is.

This sets up our instance variables and parses the arguments to the filter.

Definition at line 849 of file vf_remove_logo.c.

Variable Documentation

const vf_info_t vf_info_remove_logo

Initial value:

 {
    "Removes a tv logo based on a mask image.",
    "remove-logo",
    "Robert Edele",
    "",
    vf_open,
    NULL
}

Meta data about our filter.

Definition at line 897 of file vf_remove_logo.c.


Data Structures
struct	pgm_structure
	Simple implementation of the PGM image format. More...
struct	vf_priv_s
	Stores persistant variables. More...
Defines
#define	max(x, y) ((x)>(y)?(x):(y))
	Returns the larger of the two arguments.
#define	min(x, y) ((x)>(y)?(y):(x))
	Returns the smaller of the two arguments.
#define	test_filter(image, x, y) ((unsigned char) (image->pixel[((y) * image->width) + (x)]))
	Test if a pixel is part of the logo.
#define	apply_mask_fudge_factor(x) (((x) >> 2) + x)
	Chooses a slightly larger mask size to improve performance.
#define	REMOVE_LOGO_LOAD_PGM_ERROR_MESSAGE(message) {mp_msg(MSGT_VFILTER, MSGL_ERR, message); return NULL;}
Functions
static void *	safe_malloc (int size)
	Mallocs memory and checks to make sure it succeeded.
static void	calculate_bounding_rectangle (int posx1, int posy1, int posx2, int posy2, pgm_structure *filter)
	Calculates the smallest rectangle that will encompass the logo region.
static void	destroy_masks (vf_instance_t *vf)
	Free mask memory.
static void	initialize_masks (vf_instance_t *vf)
	Set up our array of masks.
static void	convert_mask_to_strength_mask (vf_instance_t vf, pgm_structure mask)
	Pre-processes an image to give distance information.
static void	get_blur (const vf_instance_t const vf, unsigned int const value_out, const pgm_structure const logo_mask, const mp_image_t const image, const int x, const int y, const int plane)
	Our blurring function.
static void	destroy_pgm (pgm_structure *to_be_destroyed)
	Free a pgm_structure.
static void	load_pgm_skip (FILE *f)
	Helper function for load_pgm(.
static pgm_structure *	load_pgm (const char *file_name)
	Loads a raw pgm or ppm file into a newly created pgm_structure object.
static pgm_structure *	generate_half_size_image (vf_instance_t vf, pgm_structure input_image)
	Generates a scaled down image with half width, height, and intensity.
static unsigned int	find_best (struct vf_instance *vf)
	Checks if YV12 is supported by the next filter.
static int	config (struct vf_instance *vf, int width, int height, int d_width, int d_height, unsigned int flags, unsigned int outfmt)
	Configure the filter and call the next filter's config function.
static void	convert_yv12 (const vf_instance_t const vf, const char const source, const int source_stride, const mp_image_t const source_image, const int width, const int height, char const destination, const int destination_stride, int is_image_direct, pgm_structure *filter, const int plane, const int logo_start_x, const int logo_start_y, const int logo_end_x, const int logo_end_y)
	Removes the logo from a plane (either luma or chroma).
static int	put_image (struct vf_instance vf, mp_image_t mpi, double pts)
	Process a frame.
static int	query_format (struct vf_instance *vf, unsigned int fmt)
	Checks to see if the next filter accepts YV12 images.
static void	uninit (vf_instance_t *vf)
	Frees memory that our filter allocated.
static int	vf_open (vf_instance_t vf, char args)
	Initializes our filter.
Variables
const vf_info_t	vf_info_remove_logo
	Meta data about our filter.