[FFmpeg-devel] [PATCH] pnmdec: add support for mono images with non-space-separated pixel digits

[Stefano Sabatini]

When the file to decode contains a sequence of binary values like
"1101110...", decode_frame() was reading the sequence of digits like a
unique integer value, which was resulting in integer overflow and
out-of-buffer reads.

The change add support for parsing non-space-separated pixel digits
for mono formats, in particular fix decoding of file battrace.pbm, and
fix trac issue #154.
---
 libavcodec/pnmdec.c |   14 ++++++++++----
 1 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/libavcodec/pnmdec.c b/libavcodec/pnmdec.c
index 6237e9a..53e50af 100644
--- a/libavcodec/pnmdec.c
+++ b/libavcodec/pnmdec.c
@@ -104,10 +104,16 @@ static int pnm_decode_frame(AVCodecContext *avctx, void *data,
                         s->bytestream++;
                     if(s->bytestream >= s->bytestream_end)
                         return -1;
-                    do{
-                        v= 10*v + c;
-                        c= (*s->bytestream++) - '0';
-                    }while(c <= 9);
+                    while (s->bytestream < s->bytestream_end) {
+                        c = (*s->bytestream++) - '0';
+                        if (c > 9)
+                            break;
+                        v = 10*v + c;
+                        if ((avctx->pix_fmt == PIX_FMT_MONOWHITE ||
+                             avctx->pix_fmt == PIX_FMT_MONOBLACK) &&
+                            *s->bytestream - '0' <= 9)
+                            break;
+                    }
                     put_bits(&pb, sample_len, (((1<<sample_len)-1)*v + (s->maxval>>1))/s->maxval);
                 }
                 flush_put_bits(&pb);
-- 
1.7.2.3