[FFmpeg-cvslog] avcodec/rawdec: Fix bits_per_coded_sample checks

[Michael Niedermayer]

ffmpeg | branch: release/3.1 | Michael Niedermayer <michael at niedermayer.cc> | Fri Aug 19 02:07:22 2016 +0200| [afd57722e1a8b749fc3c753824d26c2d7c0f9106] | committer: Michael Niedermayer

avcodec/rawdec: Fix bits_per_coded_sample checks

Fixes assertion failure
Fixes: 9eb9cf5b8c26dd0fa7107ed0348dcc1f/signal_sigabrt_7ffff6ae7c37_8926_4609a5c3f071d555d2d557625f9687b1.swf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 237207645b36fb79759d313c0399ee93ba467b9d)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=afd57722e1a8b749fc3c753824d26c2d7c0f9106
---

 libavcodec/rawdec.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavcodec/rawdec.c b/libavcodec/rawdec.c
index f97a839..5a98258 100644
--- a/libavcodec/rawdec.c
+++ b/libavcodec/rawdec.c
@@ -204,8 +204,9 @@ static int raw_decode(AVCodecContext *avctx, void *data, int *got_frame,
 
     desc = av_pix_fmt_desc_get(avctx->pix_fmt);
 
-    if ((avctx->bits_per_coded_sample == 8 || avctx->bits_per_coded_sample == 4
-            || avctx->bits_per_coded_sample <= 2) &&
+    if ((avctx->bits_per_coded_sample == 8 || avctx->bits_per_coded_sample == 4 ||
+         avctx->bits_per_coded_sample == 2 || avctx->bits_per_coded_sample == 1 ||
+         (avctx->bits_per_coded_sample == 0 && (context->is_nut_pal8 || context->is_mono)) ) &&
         (context->is_mono || context->is_pal8) &&
         (!avctx->codec_tag || avctx->codec_tag == MKTAG('r','a','w',' ') ||
                 context->is_nut_mono || context->is_nut_pal8)) {