I tested this using internal mp4, as well as libvpx. In both cases the pipeline completed without a glitch on a sample of 1400 videos.
This sounds vaguely familiar...f it fails with libx264 but works well with the others, you may be running up against a threading issue we're trying to resolve in the zeranoe builds. Could you try with this build and see if it works? http://x32.elijst.nl/FFmpeg-20120622.7z
Also gdb backtraces of stuck threads?
Correct me if I'm wrong, but I won't be able to get it in Windows, will I?
Believe it or not the mingw people have ported gdb to windows, and it works quite well. http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages... It can even attach to remote processes. You'll want to be running a debug version of ffmpeg, though even a non debug version of it might show some interesting material. basically start gdb, then (gdb) attach <pidd> (gdb) thread apply all bt GL!