[FFmpeg-devel] [BUG swscale] Memory Leak

[Pavel Pavlov]

I was comparing results of different algorithms and noticed that after
some amount of time I got crashes inside swscale function. I tried to
debug it and the reason for that appears that VirtualAlloc returns NULL
inside:
 
c->chrMmx2FilterCode = VirtualAlloc(NULL, c->chrMmx2FilterCodeSize,
MEM_COMMIT, PAGE_EXECUTE_READWRITE); [c->chrMmx2FilterCode is NULL,
@err is 8, not enough memory]
 
strangely, I didn't have any memory problems at all (a couple of gigs
free)! But I noticed that after a few minutes of running tests at
100%cpu my app would take like 150MB of ram (same size for virtual
memory). I tried to debug, and it appears that both VirtualFree calls
inside sws_freeContext always fail.
VirtualFree(c->lumMmx2FilterCode, c->lumMmx2FilterCodeSize,
MEM_RELEASE);  // @err=87 ERROR_INVALID_PARAMETER
VirtualFree(c->chrMmx2FilterCode, c->chrMmx2FilterCodeSize,
MEM_RELEASE);