[FFmpeg-devel] libx264.c bug?

[C Chatterjee]

Noticed an issue with the X264_frame() function in libx264.c in libavcodec in the 0.8.5 release.
The do-while() loop below:
   do {
    if (x264_encoder_encode(x4->enc, &nal, &nnal, frame? &x4->pic: NULL, &pic_out) < 0)
        return -1;

    bufsize = encode_nals(ctx, buf, bufsize, nal, nnal, 0);
    if (bufsize < 0)
        return -1;
    } while (!bufsize && !frame && x264_encoder_delayed_frames(x4->enc));

bufsize returned by encode_nals(() is the buffer used. If it does not use any buffer it returns 0. So bufsize=0 at the end of thisiteration.
Next iteration it uses bufsize=0 as an argument to encode_nals(). Clearly this is a problem.

Did I miss something?

Chanchal Chatterjee