[FFmpeg-cvslog] avcodec/xface: correct the XFACE_MAX_* values

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Tue Dec 16 18:57:54 2014 +0100| [93a5a16f136d095d23610f57bdad10ba88120fba] | committer: Michael Niedermayer

avcodec/xface: correct the XFACE_MAX_* values

Fixes out of array access

Fixes: asan_stack-oob_32c12e5_2536_cov_2442316831_lena.xface
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=93a5a16f136d095d23610f57bdad10ba88120fba
---

 libavcodec/xface.h |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/libavcodec/xface.h b/libavcodec/xface.h
index 63df5d3..0236d71 100644
--- a/libavcodec/xface.h
+++ b/libavcodec/xface.h
@@ -41,17 +41,17 @@
 /*
  * Image is encoded as a big integer, using characters from '~' to
  * '!', for a total of 94 symbols. In order to express
- * 48x48*2=8*XFACE_MAX_WORDS=4608
- * bits, we need a total of 704 digits, as given by:
- * ceil(lg_94(2^4608)) = 704
+ * 48x48 pixels with the worst case encoding 666 symbols should
+ * be sufficient.
  */
-#define XFACE_MAX_DIGITS 704
+#define XFACE_MAX_DIGITS 666
 
 #define XFACE_BITSPERWORD 8
 #define XFACE_WORDCARRY (1 << XFACE_BITSPERWORD)
 #define XFACE_WORDMASK (XFACE_WORDCARRY - 1)
 
-#define XFACE_MAX_WORDS ((XFACE_PIXELS * 2 + XFACE_BITSPERWORD - 1) / XFACE_BITSPERWORD)
+// This must be larger or equal to log256(94^XFACE_MAX_DIGITS)
+#define XFACE_MAX_WORDS 546
 
 /* Portable, very large unsigned integer arithmetic is needed.
  * Implementation uses arrays of WORDs. */