[FFmpeg-cvslog] diracdec: fix edge emulation check, fixes out of array reads

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Sun Nov 11 03:34:09 2012 +0100| [dca2fa10d37022684c61166be59294c9f98530d4] | committer: Michael Niedermayer

diracdec: fix edge emulation check, fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=dca2fa10d37022684c61166be59294c9f98530d4
---

 libavcodec/diracdec.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/diracdec.c b/libavcodec/diracdec.c
index 13d9e47..a8f5938 100644
--- a/libavcodec/diracdec.c
+++ b/libavcodec/diracdec.c
@@ -1404,8 +1404,8 @@ static int mc_subpel(DiracContext *s, DiracBlock *block, const uint8_t *src[5],
     }
 
     /* fixme: v/h _edge_pos */
-    if ((unsigned)x > p->width +EDGE_WIDTH/2 - p->xblen ||
-        (unsigned)y > p->height+EDGE_WIDTH/2 - p->yblen) {
+    if ((unsigned)x > FFMAX(p->width +EDGE_WIDTH/2 - p->xblen, 0) ||
+        (unsigned)y > FFMAX(p->height+EDGE_WIDTH/2 - p->yblen, 0)) {
         for (i = 0; i < nplanes; i++) {
             ff_emulated_edge_mc(s->edge_emu_buffer[i], src[i], p->stride,
                                 p->xblen, p->yblen, x, y,