[FFmpeg-cvslog] h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3

[Anton Khirnov]

ffmpeg | branch: release/0.10 | Anton Khirnov <anton at khirnov.net> | Thu Nov 28 10:54:35 2013 +0100| [51ae8e26af8f5b26efb41edc0fe4812368d16ae9] | committer: Reinhard Tartler

h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3

Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable at libav.org

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=51ae8e26af8f5b26efb41edc0fe4812368d16ae9
---

 libavcodec/h264.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavcodec/h264.c b/libavcodec/h264.c
index d6e1ba1..de4a4f0 100644
--- a/libavcodec/h264.c
+++ b/libavcodec/h264.c
@@ -107,10 +107,10 @@ int ff_h264_check_intra4x4_pred_mode(H264Context *h){
  */
 int ff_h264_check_intra_pred_mode(H264Context *h, int mode, int is_chroma){
     MpegEncContext * const s = &h->s;
-    static const int8_t top [7]= {LEFT_DC_PRED8x8, 1,-1,-1};
-    static const int8_t left[7]= { TOP_DC_PRED8x8,-1, 2,-1,DC_128_PRED8x8};
+    static const int8_t top[4]  = { LEFT_DC_PRED8x8, 1, -1, -1 };
+    static const int8_t left[5] = { TOP_DC_PRED8x8, -1,  2, -1, DC_128_PRED8x8 };
 
-    if(mode > 6U) {
+    if(mode > 3U) {
         av_log(h->s.avctx, AV_LOG_ERROR, "out of range intra chroma pred mode at %d %d\n", s->mb_x, s->mb_y);
         return -1;
     }