[PATCH] updated! vf_overlay alpha patch and watermarking using PNG with alpha
For those that have given up on ffmpeg and +libavfilter -vhook debacle. Here's a "just add water" solution to all your trouble: transparent PNG's and watermarks in recent ffmpeg compilations. I'm attaching a working patch that will enable alpha-transparency in overlay filter. This, on the other hand, will let you use overlay filter to superimpose a video clip or transparent png watermark on another video. This has been tested with recent revisions of svn:// svn.ffmpeg.org/soc/libavfilter rev. 5387 and ffmpeg rev. 19868 (16 Sep 2009). Compiled and running under both x86 and x86_64 on linux. Example 1 - insert transparent PNG watermark in bottom left corner of the video: -vfilters "movie=0:png:logo.png [wm];[in][wm] overlay=10:mainH-overlayH-10:1 [out]" Notice the last parameter to overlay ":1" - this enables alpha blending. Example 2 - insert 2 different transparent PNG watermarks (second watermark on bottom right corner): -vfilters "movie=0:png:logo.png [wm];movie=0:png:logo2.png [awm];[in][wm] overlay=10:mainH-overlayH-10:1 [int];[int][awm] overlay=mainW-overlayW-10:mainH-overlayH-10:1 [out]" You could chain and add more overlays this way but the efficiency of such approach is yet to be tested. This should save you a lot of time and you can continue to use your favorite transcoder for web video publishing! With best regards. -- __ /.)\ +48 695 600 936 \(./ abodera@gmail.com
Artur Bodera wrote:
For those that have given up on ffmpeg and +libavfilter -vhook debacle.
Here's a "just add water" solution to all your trouble: transparent PNG's and watermarks in recent ffmpeg compilations.
I'm attaching a working patch that will enable alpha-transparency in overlay filter. This, on the other hand, will let you use overlay filter to superimpose a video clip or transparent png watermark on another video. This has been tested with recent revisions of svn:// svn.ffmpeg.org/soc/libavfilter rev. 5387 and ffmpeg rev. 19868 (16 Sep 2009). Compiled and running under both x86 and x86_64 on linux.
Example 1 - insert transparent PNG watermark in bottom left corner of the video: -vfilters "movie=0:png:logo.png [wm];[in][wm] overlay=10:mainH-overlayH-10:1 [out]"
Notice the last parameter to overlay ":1" - this enables alpha blending.
Example 2 - insert 2 different transparent PNG watermarks (second watermark on bottom right corner): -vfilters "movie=0:png:logo.png [wm];movie=0:png:logo2.png [awm];[in][wm] overlay=10:mainH-overlayH-10:1 [int];[int][awm] overlay=mainW-overlayW-10:mainH-overlayH-10:1 [out]"
You could chain and add more overlays this way but the efficiency of such approach is yet to be tested.
This should save you a lot of time and you can continue to use your favorite transcoder for web video publishing!
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this? -Vitor
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded:
Artur Bodera wrote:
For those that have given up on ffmpeg and +libavfilter -vhook debacle.
Here's a "just add water" solution to all your trouble: transparent PNG's and watermarks in recent ffmpeg compilations.
I'm attaching a working patch that will enable alpha-transparency in overlay filter. This, on the other hand, will let you use overlay filter to superimpose a video clip or transparent png watermark on another video. This has been tested with recent revisions of svn:// svn.ffmpeg.org/soc/libavfilter rev. 5387 and ffmpeg rev. 19868 (16 Sep 2009). Compiled and running under both x86 and x86_64 on linux.
Example 1 - insert transparent PNG watermark in bottom left corner of the video: -vfilters "movie=0:png:logo.png [wm];[in][wm] overlay=10:mainH-overlayH-10:1 [out]"
Notice the last parameter to overlay ":1" - this enables alpha blending.
Example 2 - insert 2 different transparent PNG watermarks (second watermark on bottom right corner): -vfilters "movie=0:png:logo.png [wm];movie=0:png:logo2.png [awm];[in][wm] overlay=10:mainH-overlayH-10:1 [int];[int][awm] overlay=mainW-overlayW-10:mainH-overlayH-10:1 [out]"
You could chain and add more overlays this way but the efficiency of such approach is yet to be tested.
This should save you a lot of time and you can continue to use your favorite transcoder for web video publishing!
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this? Regards.
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded: [...] I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
A somehow tinied-up version which can be used as a base for SVN inclusion, I believe we should get rid of the blend parameter, also the expression evaluation may stay in a successive commit. Regards.
Stefano Sabatini wrote:
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded: [...] I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
A somehow tinied-up version which can be used as a base for SVN inclusion,
Nice work!
I believe we should get rid of the blend parameter, also the expression evaluation may stay in a successive commit.
Do you have any idea of how to get rid of it? Are you planning to implement Michael's suggestion of calling swscale to convert one of the inputs to the right pixel format? A few comments...
static int query_formats(AVFilterContext *ctx) { OverlayContext *over = ctx->priv;
if (over->blend) { enum PixelFormat pix_fmts1[] = { PIX_FMT_YUV420P, PIX_FMT_NONE }; enum PixelFormat pix_fmts2[] = { PIX_FMT_YUVA420P, PIX_FMT_NONE };
these can be const.
} else { avfilter_default_query_formats(ctx);
Are all lavfi formats supported?
static void copy_blended( uint8_t *out, int out_linesize, const uint8_t *in, int in_linesize, const uint8_t *alpha, int alpha_linesize, int w, int h, int hsub, int vsub) { int x, y;
for (y = 0; y < h; y++) { uint8_t *outp = out + y * out_linesize; const uint8_t *inp = in + y * in_linesize; const uint8_t *alphap = alpha + (y<<vsub) * alpha_linesize;
for (x = 0; x < w; x++) { *outp = (*outp * (0xff - *alphap) + *inp * *alphap) >> 8;
Looks like some rounding could be useful. Also, one thing that bothers me is that this filter copies the whole picture for overlaying a 10x10 picture into a 1024x1024 picture. I think it could simply draw the input #2 over input #1 (like vf_drawbox does with its box). -Vitor
Hi Vitor, On 12/08/2009 10:37 AM, Vitor Sessak wrote:
Stefano Sabatini wrote:
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded: [...] I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
A somehow tinied-up version which can be used as a base for SVN inclusion,
Nice work!
I believe we should get rid of the blend parameter, also the expression evaluation may stay in a successive commit.
Do you have any idea of how to get rid of it? Are you planning to implement Michael's suggestion of calling swscale to convert one of the inputs to the right pixel format?
A few comments...
static int query_formats(AVFilterContext *ctx) { OverlayContext *over = ctx->priv;
if (over->blend) { enum PixelFormat pix_fmts1[] = { PIX_FMT_YUV420P, PIX_FMT_NONE }; enum PixelFormat pix_fmts2[] = { PIX_FMT_YUVA420P, PIX_FMT_NONE };
these can be const.
} else { avfilter_default_query_formats(ctx);
Are all lavfi formats supported?
static void copy_blended( uint8_t *out, int out_linesize, const uint8_t *in, int in_linesize, const uint8_t *alpha, int alpha_linesize, int w, int h, int hsub, int vsub) { int x, y;
for (y = 0; y < h; y++) { uint8_t *outp = out + y * out_linesize; const uint8_t *inp = in + y * in_linesize; const uint8_t *alphap = alpha + (y<<vsub) * alpha_linesize;
for (x = 0; x < w; x++) { *outp = (*outp * (0xff - *alphap) + *inp * *alphap) >> 8;
Looks like some rounding could be useful.
Like: outp = (*outp * (0xff - *alphap) + *inp * *alphap + 128) >> 8; ? -- Baptiste COUDURIER Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA FFmpeg maintainer http://www.ffmpeg.org
Baptiste Coudurier wrote:
Hi Vitor,
On 12/08/2009 10:37 AM, Vitor Sessak wrote:
Stefano Sabatini wrote:
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded: [...] I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
A somehow tinied-up version which can be used as a base for SVN inclusion,
Nice work!
I believe we should get rid of the blend parameter, also the expression evaluation may stay in a successive commit.
Do you have any idea of how to get rid of it? Are you planning to implement Michael's suggestion of calling swscale to convert one of the inputs to the right pixel format?
A few comments...
static int query_formats(AVFilterContext *ctx) { OverlayContext *over = ctx->priv;
if (over->blend) { enum PixelFormat pix_fmts1[] = { PIX_FMT_YUV420P, PIX_FMT_NONE }; enum PixelFormat pix_fmts2[] = { PIX_FMT_YUVA420P, PIX_FMT_NONE };
these can be const.
} else { avfilter_default_query_formats(ctx);
Are all lavfi formats supported?
static void copy_blended( uint8_t *out, int out_linesize, const uint8_t *in, int in_linesize, const uint8_t *alpha, int alpha_linesize, int w, int h, int hsub, int vsub) { int x, y;
for (y = 0; y < h; y++) { uint8_t *outp = out + y * out_linesize; const uint8_t *inp = in + y * in_linesize; const uint8_t *alphap = alpha + (y<<vsub) * alpha_linesize;
for (x = 0; x < w; x++) { *outp = (*outp * (0xff - *alphap) + *inp * *alphap) >> 8;
Looks like some rounding could be useful.
Like: outp = (*outp * (0xff - *alphap) + *inp * *alphap + 128) >> 8;
?
I think so. -Vitor
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded:
Artur Bodera wrote:
For those that have given up on ffmpeg and +libavfilter -vhook debacle.
Here's a "just add water" solution to all your trouble: transparent PNG's and watermarks in recent ffmpeg compilations.
I'm attaching a working patch that will enable alpha-transparency in overlay filter. This, on the other hand, will let you use overlay filter to superimpose a video clip or transparent png watermark on another video. This has been tested with recent revisions of svn:// svn.ffmpeg.org/soc/libavfilter rev. 5387 and ffmpeg rev. 19868 (16 Sep 2009). Compiled and running under both x86 and x86_64 on linux.
Example 1 - insert transparent PNG watermark in bottom left corner of the video: -vfilters "movie=0:png:logo.png [wm];[in][wm] overlay=10:mainH-overlayH-10:1 [out]"
Notice the last parameter to overlay ":1" - this enables alpha blending.
Example 2 - insert 2 different transparent PNG watermarks (second watermark on bottom right corner): -vfilters "movie=0:png:logo.png [wm];movie=0:png:logo2.png [awm];[in][wm] overlay=10:mainH-overlayH-10:1 [int];[int][awm] overlay=mainW-overlayW-10:mainH-overlayH-10:1 [out]"
You could chain and add more overlays this way but the efficiency of such approach is yet to be tested.
This should save you a lot of time and you can continue to use your favorite transcoder for web video publishing!
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
I'll apply the patch soon as this feature is probably the most requested for lavfi, then I'll work on a proper filter integration when the lavfi test system will be ready (I hope within a month). Regards.
On date Wednesday 2010-02-03 19:56:12 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded:
Artur Bodera wrote: [...]
This should save you a lot of time and you can continue to use your favorite transcoder for web video publishing!
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
I'll apply the patch soon as this feature is probably the most requested for lavfi, then I'll work on a proper filter integration when the lavfi test system will be ready (I hope within a month).
Applied, hope this will raise the interest of some wanna-bee contributor ;-)!
On 2/3/10 4:55 PM, Stefano Sabatini wrote:
On date Wednesday 2010-02-03 19:56:12 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded:
Artur Bodera wrote: [...]
This should save you a lot of time and you can continue to use your favorite transcoder for web video publishing!
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
I'll apply the patch soon as this feature is probably the most requested for lavfi, then I'll work on a proper filter integration when the lavfi test system will be ready (I hope within a month).
Applied, hope this will raise the interest of some wanna-bee contributor ;-)!
Stefano, can you please apply your cleanup patch in soc svn ? I intend to work on this. -- Baptiste COUDURIER Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA FFmpeg maintainer http://www.ffmpeg.org
On date Thursday 2010-04-29 22:23:38 -0700, Baptiste Coudurier encoded:
On 2/3/10 4:55 PM, Stefano Sabatini wrote:
On date Wednesday 2010-02-03 19:56:12 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:57:59 +0100, Stefano Sabatini encoded:
On date Tuesday 2009-12-01 00:50:50 +0100, Vitor Sessak encoded:
Artur Bodera wrote: [...]
This should save you a lot of time and you can continue to use your favorite transcoder for web video publishing!
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I'm fine with committing the patch to soc if tested, even better would be to try to push the filter to the main repo. Does someone want to volunteer for this?
I'll apply the patch soon as this feature is probably the most requested for lavfi, then I'll work on a proper filter integration when the lavfi test system will be ready (I hope within a month).
Applied, hope this will raise the interest of some wanna-bee contributor ;-)!
Stefano, can you please apply your cleanup patch in soc svn ?
I should find the time to do it tomorrow (feel free to apply it yourself if you're in a hurry).
I intend to work on this.
Cool :-). Regards.
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format. So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense. This would also help keep the code clean! Because the whole overlay concept is very universal, there is no need for vf_logo or any other strange mockup. You can implement all the vf_logo functionality with overlay+scale. As a convenience, a mockup vf_logo could be built - which would just be a shortcut for automatically splitting and inserting vf_overlay in the right spot (just like scale or format filter is inserted automatically). This would mean the core code is still in vf_overlay and is easier to maintain in the long run. Arthur. -- __ /.)\ +48 695 600 936 \(./ abodera@gmail.com
Hi and sorry for the delay. Artur Bodera wrote:
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format.
So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense.
I agree that this would be nice, but there is no way to make it work with the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been.
Because the whole overlay concept is very universal, there is no need for vf_logo or any other strange mockup. You can implement all the vf_logo functionality with overlay+scale. As a convenience, a mockup vf_logo could be built - which would just be a shortcut for automatically splitting and inserting vf_overlay in the right spot (just like scale or format filter is inserted automatically). This would mean the core code is still in vf_overlay and is easier to maintain in the long run.
I agree completely. -Vitor
On Sun, Dec 06, 2009 at 05:12:37PM +0100, Vitor Sessak wrote:
Hi and sorry for the delay.
Artur Bodera wrote:
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format. So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense.
I agree that this would be nice, but there is no way to make it work with the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been.
an overlay filter that supports rgb -> rgb yuv -> yuv ^ OR ^ rgba yuva cant be done, but you can do: rgb -> rgb yuv -> yuv ^ OR ^ yuva||rgba yuva||rgba this might end up requireing rgb<->yuv converting the thing to be overlaid in the overlay filter (its just a call to the swscaler ...) and about the current format negotiation, if you have better ideas which i doubt ;) they are welcome. The issue is just that with arbitrary graphs its not so easy to negotiate the format in a way that is scaleable and doesnt break down to O(2^n) time [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In fact, the RIAA has been known to suggest that students drop out of college or go to community college in order to be able to afford settlements. -- The RIAA
On Mon, Dec 7, 2009 at 2:08 PM, Michael Niedermayer <michaelni@gmx.at>wrote:
The issue is just that with arbitrary graphs its not so easy to negotiate the format in a way that is scaleable and doesnt break down to O(2^n) time
Just a friendly, novice, out-of-the-box suggestion from my side - how does DirectX / DirectShow handle that? Isn't it a common scenario or a common problem that has already been resolved years ago? I remember playing with directshow filter graphs that were, in their complexity, way beyond what we're considering here. Arthur. -- __ /.)\ +48 695 600 936 \(./ abodera@gmail.com
On Mon, Dec 07, 2009 at 03:59:12PM +0100, Artur Bodera wrote:
On Mon, Dec 7, 2009 at 2:08 PM, Michael Niedermayer <michaelni@gmx.at>wrote:
The issue is just that with arbitrary graphs its not so easy to negotiate the format in a way that is scaleable and doesnt break down to O(2^n) time
Just a friendly, novice, out-of-the-box suggestion from my side - how does DirectX / DirectShow handle that?
I dont know, but if someone knows, iam interrested in that as well
Isn't it a common scenario or a common problem that has already been resolved years ago?
no, NP = P hasnt been proofen yet
I remember playing with directshow filter graphs that were, in their complexity, way beyond what we're considering here.
The problem is not the graph complexity the problem is finding an optimal solution with few format convertions. You can trivially solve any graph by just inserting a format convertion filter between any 2 filters What our limitation allows us is finding a solution without convertion filters if such a solution exists no matter how complex the graph. And the overlay filter is the first that would be a little more convenient if that limitation didnt exist. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Rewriting code that is poorly written but fully understood is good. Rewriting code that one doesnt understand is a sign that one is less smart then the original author, trying to rewrite it will not make it better.
On Mon, Dec 07, 2009 at 09:33:42PM +0100, Michael Niedermayer wrote:
On Mon, Dec 07, 2009 at 03:59:12PM +0100, Artur Bodera wrote:
On Mon, Dec 7, 2009 at 2:08 PM, Michael Niedermayer <michaelni@gmx.at>wrote:
The issue is just that with arbitrary graphs its not so easy to negotiate the format in a way that is scaleable and doesnt break down to O(2^n) time
Just a friendly, novice, out-of-the-box suggestion from my side - how does DirectX / DirectShow handle that?
I dont know, but if someone knows, iam interrested in that as well
I don't really know for sure, but I think DirectShow does not have any scale/colourspace conversion filters, at least not by default, and a lot of codecs implement internal conversions due to this. Which makes me suspect they haven't actually solved it. And I guess a few things are done via filter priorities in DirectShow, with a good chance of breaking if they are set wrong - at least in the Win98 times or so.
Michael Niedermayer wrote:
On Sun, Dec 06, 2009 at 05:12:37PM +0100, Vitor Sessak wrote:
Hi and sorry for the delay.
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format. So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense. I agree that this would be nice, but there is no way to make it work with
Artur Bodera wrote: the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been.
an overlay filter that supports rgb -> rgb yuv -> yuv ^ OR ^ rgba yuva
cant be done, but you can do:
rgb -> rgb yuv -> yuv ^ OR ^ yuva||rgba yuva||rgba
this might end up requireing rgb<->yuv converting the thing to be overlaid in the overlay filter (its just a call to the swscaler ...)
I thought about this idea, but what I don't like is doing calls to swscale instead of just having the vf_overlay inserting an scaler filter. Having a filter needing to force the insertion of others will be possible in several other cases (padding, deinterlacing, resizing, etc). That's why I thought that having at a first time a simpler solution is better (instead of delaying too much the svn inclusion). -Vitor
On Mon, Dec 07, 2009 at 07:51:35PM +0100, Vitor Sessak wrote:
Michael Niedermayer wrote:
Hi and sorry for the delay.
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format. So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense. I agree that this would be nice, but there is no way to make it work with
Artur Bodera wrote: the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been. an overlay filter that supports rgb -> rgb yuv -> yuv ^ OR ^ rgba yuva cant be done, but you can do: rgb -> rgb yuv -> yuv ^ OR ^ yuva||rgba yuva||rgba
On Sun, Dec 06, 2009 at 05:12:37PM +0100, Vitor Sessak wrote: this might end up requireing rgb<->yuv converting the thing to be overlaid in the overlay filter (its just a call to the swscaler ...)
I thought about this idea, but what I don't like is doing calls to swscale instead of just having the vf_overlay inserting an scaler filter.
well, inserting a scale filter is nicer sure ...
Having a filter needing to force the insertion of others will be possible in several other cases (padding, deinterlacing, resizing, etc).
i dont understand this sentance
That's why I thought that having at a first time a simpler solution is better (instead of delaying too much the svn inclusion).
iam not against a simple solution at first at all (the obvious one would be to support just rgb+rgba or just yuv+yuva) [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The worst form of inequality is to try to make unequal things equal. -- Aristotle
On 12/06/2009 08:12 AM, Vitor Sessak wrote:
Hi and sorry for the delay.
Artur Bodera wrote:
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format.
So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense.
I agree that this would be nice, but there is no way to make it work with the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been.
How much harm does doing yuv -> yuva or rgb -> rgba in all cases ? To my knowledge it would only be a matter of adding one component and memset it to 0xff. Libswscale should be pretty fast at doing this. -- Baptiste COUDURIER Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA FFmpeg maintainer http://www.ffmpeg.org
Baptiste Coudurier wrote:
On 12/06/2009 08:12 AM, Vitor Sessak wrote:
Hi and sorry for the delay.
Artur Bodera wrote:
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format.
So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense.
I agree that this would be nice, but there is no way to make it work with the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been.
How much harm does doing yuv -> yuva or rgb -> rgba in all cases ? To my knowledge it would only be a matter of adding one component and memset it to 0xff.
This wouldn't do much harm, but there is no way of asking for such kind of inputs in lavfi (i.e., either "rgb,rgba" or "yuv,yuva"). And changing lavfi to accept such constraints would probably make colospace negotiation a NP problem. -Vitor
On Wed, May 12, 2010 at 09:00:46AM +0200, Vitor Sessak wrote:
Baptiste Coudurier wrote:
On 12/06/2009 08:12 AM, Vitor Sessak wrote:
Hi and sorry for the delay.
Artur Bodera wrote:
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak <vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format.
So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense.
I agree that this would be nice, but there is no way to make it work with the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been. How much harm does doing yuv -> yuva or rgb -> rgba in all cases ? To my knowledge it would only be a matter of adding one component and memset it to 0xff.
This wouldn't do much harm, but there is no way of asking for such kind of inputs in lavfi (i.e., either "rgb,rgba" or "yuv,yuva"). And changing lavfi to accept such constraints would probably make colospace negotiation a NP problem.
finding a solution with a minimum of convertion could become NP, finding any solution wont heres a quick option (i do not know how good / bad this suggestion is, if someone like loren could look at this before its implemented this would probably be a good idea) 1. we extend query_format to query_format(int alternative) where alternative selects which of several (small number normally and for most filters just 1) alternative colorspace lists each link supports. Euch such alternative is like the current case that is 2 links either must have the same list of possible pixelformats/colorspaces or they are independant. 2. we run the current avfiltergraph.c query_formats() over the graph skiping all filters that have alternatives. 3. we go over all filters that have alternatives for each such filter we find the 2 best alterantives, best based on score= (A<<32) + B; A= the number of scale filters the specific alterantive would require B= the number of degrees of freedom lost (aka fewer colorspaces) we then calculate a score2 that is the difference of the scores of the best 2 alterantives. And last lock in filter alterantives iteratively based on best score2 The idea here is that we lock in the filter with the most clearly supperior alterantive first. That is if we have a filter that has 2 alterantives and neither on its own would require an additional convertion filter and another filter that also has 2 alternatives and one of these would require 1 convertion and the other case would require 2 convertions then the 1 convertion alternative would be locked in and we would restart from the begin This is not optimal, not even for a simple linear filter chain with 2 filters. But it might be an acceptable heuristic for real filter graphs. For a single linear chain its possible to find the optimal case with viterbi Another method would be to only use avfiltergraph.c query_formats() and consider all the alternative values inputs to it and the number of convertions and degrees of freedom left over the whole graph its score. And then use some generic optimization technique that minimizes this score. btw, off topic a little but simply favoring the first colorspace in each list like its done in svn is not a good idea, it likely would make more sense to favor a colorspace that preserves all information that is there and is simple. that is if either upstream or downstream is greyscale we shouldnt select yuv similar issues exist with 16/8 bit and rgb/yuv and alpha [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB It is not what we do, but why we do it that matters.
Hi, On 05/12/2010 02:33 PM, Michael Niedermayer wrote:
On Wed, May 12, 2010 at 09:00:46AM +0200, Vitor Sessak wrote:
Baptiste Coudurier wrote:
On 12/06/2009 08:12 AM, Vitor Sessak wrote:
Hi and sorry for the delay.
Artur Bodera wrote:
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak<vitor1001@gmail.com> wrote:
While I normally oppose making non-committed code more complex, I think this feature is so often requested that it is worth the extra work in the future. Stefano, Michael, any strong opinion about this?
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format.
So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense.
I agree that this would be nice, but there is no way to make it work with the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been. How much harm does doing yuv -> yuva or rgb -> rgba in all cases ? To my knowledge it would only be a matter of adding one component and memset it to 0xff.
This wouldn't do much harm, but there is no way of asking for such kind of inputs in lavfi (i.e., either "rgb,rgba" or "yuv,yuva"). And changing lavfi to accept such constraints would probably make colospace negotiation a NP problem.
finding a solution with a minimum of convertion could become NP, finding any solution wont
heres a quick option (i do not know how good / bad this suggestion is, if someone like loren could look at this before its implemented this would probably be a good idea)
1. we extend query_format to query_format(int alternative) where alternative selects which of several (small number normally and for most filters just 1) alternative colorspace lists each link supports. Euch such alternative is like the current case that is 2 links either must have the same list of possible pixelformats/colorspaces or they are independant.
2. we run the current avfiltergraph.c query_formats() over the graph skiping all filters that have alternatives.
3. we go over all filters that have alternatives for each such filter we find the 2 best alterantives, best based on score= (A<<32) + B; A= the number of scale filters the specific alterantive would require B= the number of degrees of freedom lost (aka fewer colorspaces) we then calculate a score2 that is the difference of the scores of the best 2 alterantives. And last lock in filter alterantives iteratively based on best score2
The idea here is that we lock in the filter with the most clearly supperior alterantive first. That is if we have a filter that has 2 alterantives and neither on its own would require an additional convertion filter and another filter that also has 2 alternatives and one of these would require 1 convertion and the other case would require 2 convertions then the 1 convertion alternative would be locked in and we would restart from the begin
This is not optimal, not even for a simple linear filter chain with 2 filters. But it might be an acceptable heuristic for real filter graphs. For a single linear chain its possible to find the optimal case with viterbi
Another method would be to only use avfiltergraph.c query_formats() and consider all the alternative values inputs to it and the number of convertions and degrees of freedom left over the whole graph its score. And then use some generic optimization technique that minimizes this score.
btw, off topic a little but simply favoring the first colorspace in each list like its done in svn is not a good idea, it likely would make more sense to favor a colorspace that preserves all information that is there and is simple. that is if either upstream or downstream is greyscale we shouldnt select yuv similar issues exist with 16/8 bit and rgb/yuv and alpha
Question, why not using a pull paradigm from the output filter ? That certainly gives an hint on what the user want at the end. This would allow the overlay filter to request RGBA as input when output is RGB, same for YUVA/YUV -- Baptiste COUDURIER Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA FFmpeg maintainer http://www.ffmpeg.org
On Thu, May 13, 2010 at 01:05:58PM -0700, Baptiste Coudurier wrote:
Hi,
On 05/12/2010 02:33 PM, Michael Niedermayer wrote:
On Wed, May 12, 2010 at 09:00:46AM +0200, Vitor Sessak wrote:
Baptiste Coudurier wrote:
On 12/06/2009 08:12 AM, Vitor Sessak wrote:
Hi and sorry for the delay.
Artur Bodera wrote:
On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak<vitor1001@gmail.com> wrote:
> While I normally oppose making non-committed code more complex, I > think > this feature is so often requested that it is worth the extra work in > the > future. Stefano, Michael, any strong opinion about this? >
I think the vf_overlay should be modified altogether. Although mathematically alpha-blending is more expensive than opaque pixel replacement, I think that it should be automatically decided by analyzing the overlay format.
So the alpha-blending should be a "built-in" functionality (not a switchable parameter) and should be implicitly functional with any overlay stream/image that has alpha channel (i.e. rgba). If there is no alpha channel, then pixel overriding would be used. This makes much more sense.
I agree that this would be nice, but there is no way to make it work with the current format negotiation in libavfilter. For example, there is no way to have a filter that accepts either "input: rgb, output rgba" or "input: yuv, output: yuva", so I suggest you just do as your present patch for the time been. How much harm does doing yuv -> yuva or rgb -> rgba in all cases ? To my knowledge it would only be a matter of adding one component and memset it to 0xff.
This wouldn't do much harm, but there is no way of asking for such kind of inputs in lavfi (i.e., either "rgb,rgba" or "yuv,yuva"). And changing lavfi to accept such constraints would probably make colospace negotiation a NP problem.
finding a solution with a minimum of convertion could become NP, finding any solution wont
heres a quick option (i do not know how good / bad this suggestion is, if someone like loren could look at this before its implemented this would probably be a good idea)
1. we extend query_format to query_format(int alternative) where alternative selects which of several (small number normally and for most filters just 1) alternative colorspace lists each link supports. Euch such alternative is like the current case that is 2 links either must have the same list of possible pixelformats/colorspaces or they are independant.
2. we run the current avfiltergraph.c query_formats() over the graph skiping all filters that have alternatives.
3. we go over all filters that have alternatives for each such filter we find the 2 best alterantives, best based on score= (A<<32) + B; A= the number of scale filters the specific alterantive would require B= the number of degrees of freedom lost (aka fewer colorspaces) we then calculate a score2 that is the difference of the scores of the best 2 alterantives. And last lock in filter alterantives iteratively based on best score2
The idea here is that we lock in the filter with the most clearly supperior alterantive first. That is if we have a filter that has 2 alterantives and neither on its own would require an additional convertion filter and another filter that also has 2 alternatives and one of these would require 1 convertion and the other case would require 2 convertions then the 1 convertion alternative would be locked in and we would restart from the begin
This is not optimal, not even for a simple linear filter chain with 2 filters. But it might be an acceptable heuristic for real filter graphs. For a single linear chain its possible to find the optimal case with viterbi
Another method would be to only use avfiltergraph.c query_formats() and consider all the alternative values inputs to it and the number of convertions and degrees of freedom left over the whole graph its score. And then use some generic optimization technique that minimizes this score.
btw, off topic a little but simply favoring the first colorspace in each list like its done in svn is not a good idea, it likely would make more sense to favor a colorspace that preserves all information that is there and is simple. that is if either upstream or downstream is greyscale we shouldnt select yuv similar issues exist with 16/8 bit and rgb/yuv and alpha
Question, why not using a pull paradigm from the output filter ?
Its not enough on its own. it has exponential complexity per graph size for some filter graphs. the issue is that a filter with multiple inputs cant just pull each idenpandantly. They might very well have a common source filter or a feedback path which can lead to complex dependancies. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you think the mosad wants you dead since a long time then you are either wrong or dead since a long time.
participants (6)
-
Artur Bodera -
Baptiste Coudurier -
Michael Niedermayer -
Reimar Döffinger -
Stefano Sabatini -
Vitor Sessak