[FFmpeg-devel] [PATCH]lavf/webpenc: Do not set alpha flag for yuv420p

[Carl Eugen Hoyos]

On Tuesday 03 May 2016 06:11:59 pm Ronald S. Bultje wrote:
> Hi,
>
> On Tue, May 3, 2016 at 11:54 AM, Carl Eugen Hoyos <cehoyos at ag.or.at> wrote:
> > On Tuesday 03 May 2016 03:04:50 pm Ronald S. Bultje wrote:
> > > > +                if (av_pix_fmt_desc_get(st->codecpar->format)->flags
> > > > && AV_PIX_FMT_FLAG_ALPHA)
> > >
> > > I think you want s/&&/&/?
> >
> > Definitely, new patch attached.
>
> So, to address wm4's other concern, are there cases where
> st->codecpar->format is not set? E.g. codec copy or so?

I have no idea but attached covers this case.

Carl Eugen
-------------- next part --------------
From a45e3ca418dec9350c99b786e5ebc1dfb7483014 Mon Sep 17 00:00:00 2001
From: Carl Eugen Hoyos <cehoyos at ag.or.at>
Date: Tue, 3 May 2016 18:27:27 +0200
Subject: [PATCH] lavf/webpenc: Do not set alpha flag for yuv420p.

Fixes one usecase of ticket #5431.
---
 libavformat/webpenc.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/libavformat/webpenc.c b/libavformat/webpenc.c
index 2e0147c..2aecdb6 100644
--- a/libavformat/webpenc.c
+++ b/libavformat/webpenc.c
@@ -21,6 +21,7 @@
 
 #include "libavutil/intreadwrite.h"
 #include "libavutil/opt.h"
+#include "libavutil/pixdesc.h"
 #include "avformat.h"
 #include "internal.h"
 
@@ -107,7 +108,10 @@ static int flush(AVFormatContext *s, int trailer, int64_t pts)
         if (w->frame_count == 1) {
             if (!trailer) {
                 vp8x = 1;
-                flags |= 2 + 16;
+                flags |= 2;
+                if (   st->codecpar->format == AV_PIX_FMT_NONE
+                    || av_pix_fmt_desc_get(st->codecpar->format)->flags & AV_PIX_FMT_FLAG_ALPHA)
+                    flags |= 16; // stream may contain transparency information
             }
 
             if (vp8x) {
-- 
1.7.10.4