[Libav-user] Muxing MPEG-4 motion imagery into a MPEG-2 Transport Stream

[Mark Fonnemann]

I recently posted a question about Muxing MPEG-4 video into a MPEG-2 TS. I was using quite an outdated version of libav so i am assuming that is the reason I received no responses. I have rewritten the code to use release 0.8.

I am now able to write data to the MPEG-2 TS but when opening it with VLC nothing appears despite the fact it knows there is a MPEG-4 video stream in there. I then tried to inspect with ffmpeg as follows:

> ffmpeg.exe -i c:\bat.ts

ffmpeg version 0.8, Copyright (c) 2000-2011 the FFmpeg developers
  built on Jul 23 2011 23:24:23 with gcc 3.4.4 (cygming special, gdc 0.12, using
 dmd 0.125)
  configuration: --enable-memalign-hack --enable-swscale --enable-w32threads --e
nable-shared --target-os=mingw32 --extra-cflags=-mno-cygwin --extra-libs=-mno-cy
gwin
  libavutil    51.  9. 1 / 51.  9. 1
  libavcodec   53.  7. 0 / 53.  7. 0
  libavformat  53.  4. 0 / 53.  4. 0
  libavdevice  53.  1. 1 / 53.  1. 1
  libavfilter   2. 23. 0 /  2. 23. 0
  libswscale    2.  0. 0 /  2.  0. 0
[mpeg4 @ 00ba80b0] hmm, seems the headers are not complete, trying to guess time_increment_bits
[mpeg4 @ 00ba80b0] my guess is 5 bits ;)
Compiler did not align stack variables. Libavcodec has been miscompiled
and may be very slow or crash. This is not a bug in libavcodec,
but in the compiler. You may try recompiling using gcc >= 4.2.
Do not report crashes to FFmpeg developers.
[mpeg4 @ 00ba80b0] hmm, seems the headers are not complete, trying to guess time
_increment_bits
[mpeg4 @ 00ba80b0] my guess is 5 bits ;)
[mpeg4 @ 00ba80b0] looks like this file was encoded with (divx4/(old)xvid/opendivx) -> forcing low_delay flag
[mpeg4 @ 00ba80b0] [IMGUTILS @ 0022e290] Picture size 0x0 is invalid
[mpeg4 @ 00ba80b0] get_buffer() failed (-1 0 0 00000000)
[mpeg4 @ 00ba80b0] [IMGUTILS @ 0022e290] Picture size 0x0 is invalid
[mpeg4 @ 00ba80b0] get_buffer() failed (-1 0 0 00000000)
[mpeg4 @ 00ba80b0] [IMGUTILS @ 0022e290] Picture size 0x0 is invalid
[mpeg4 @ 00ba80b0] get_buffer() failed (-1 0 0 00000000)
[mpeg4 @ 00ba80b0] [IMGUTILS @ 0022e290] Picture size 0x0 is invalid
[mpeg4 @ 00ba80b0] get_buffer() failed (-1 0 0 00000000)
[mpeg4 @ 00ba80b0] [IMGUTILS @ 0022e290] Picture size 0x0 is invalid
[mpeg4 @ 00ba80b0] get_buffer() failed (-1 0 0 00000000)
[mpeg4 @ 00ba80b0] [IMGUTILS @ 0022e290] Picture size 0x0 is invalid
[mpeg4 @ 00ba80b0] get_buffer() failed (-1 0 0 00000000)
[mpeg4 @ 00ba80b0] [IMGUTILS @ 0022e290] Picture size 0x0 is invalid
[mpeg4 @ 00ba80b0] get_buffer() failed (-1 0 0 00000000)
[mpegts @ 003ea370] Could not find codec parameters (Video: mpeg4, yuv420p)
[NULL @ 00ba80b0] start time is not set in av_estimate_timings_from_pts
c:\bat.ts: could not find codec parameters

any help would be greatly appreciated.

mark.

p.s. here's some code that i am using:

void initialize()
{
	int framewidth = 704;
	int frameheight = 480;
	
	//av_register_all etc.
        init();

	// alloc the libavformat context and set the type to "MPEG-2 TS"
	avformat_alloc_output_context2(&muxer, NULL, "mpegts", NULL);
	if(!muxer) {
		// error handling	
        }

	// add the video stream to the muxer
	addVideoStream(codec_name, framewidth, frameheight);

	// not sure what this does? not documented in doxygen
	av_dump_format(muxer, 0, "c:\\bat.ts", 1);

	// open the video stream and allocate default buffers
	openVideoStream();

	// allocate the buffer to write the destination packets to
	avio_open(&muxer->pb, "c:\\bat.ts", URL_WRONLY);

	// set the parameters for the muxer
	av_write_header(muxer);
}

void addVideoStream(std::string codec_name, int width, int height)
{
	// add a new stream to the container
	vstream = av_new_stream(muxer, 0);
	if (!vstream) {
		// error handling
	}

	// set a handle to the stream's codec ("the codec context")
	codecContext = vstream->codec;

	// set the stream type to video and specify the codec
	codecContext->codec_type = AVMEDIA_TYPE_VIDEO;

	if ( "H.264" == codec_name) 
	{
		codecContext->codec_id = CODEC_ID_H264;
	}
	else if ( "MPEG-4" == codec_name) 
	{
		codecContext->codec_id = CODEC_ID_MPEG4;
	} 
	else // "MPEG2"
	{
		codecContext->codec_id = CODEC_ID_MPEG2VIDEO;
	}

	// fill in sample parameters
	codecContext->global_quality = FF_QP2LAMBDA;
	codecContext->bit_rate = 200000;

	// resolution must be a multiple of two
	codecContext->width = width;  
	codecContext->height = height;
	
	// set additional properties of the video stream
	codecContext->time_base.den = 25;  
	codecContext->time_base.num = 1;
	codecContext->gop_size = 12; /* emit one intra frame every twelve frames at most */
	codecContext->pix_fmt = PIX_FMT_YUV420P;
	codecContext->flags |= CODEC_FLAG_GLOBAL_HEADER;
	codecContext->flags |= CODEC_FLAG_QSCALE;
	codecContext->max_b_frames = 0;

	// some formats want stream headers to be separate
	if (muxer->oformat->flags & AVFMT_GLOBALHEADER)
		codecContext->flags |= CODEC_FLAG_GLOBAL_HEADER;
}

void openVideoStream()
{
	// get the encoder for the codec
	vcodec = avcodec_find_encoder(codecContext->codec_id);

	// attempt to open the encoder
	if (avcodec_open(codecContext, vcodec) < 0)
	{
		// error handling
	}

	video_outbuf_size = 200000; // not sure how this number was chosen? it seems to be the accepted value though
	fTo = (uint8_t *) malloc(video_outbuf_size);
	video_outbuf = (uint8_t *) malloc(video_outbuf_size);

}


void doGetNextFrame(AVFrame* rgbimage)
{

	AVFrame *tmp_picture = avcodec_alloc_frame();
	unsigned int size = avpicture_get_size(codecContext->pix_fmt, codecContext->width, codecContext->height);
	uint8_t *picture_buf = (uint8_t *) malloc(size);

	int fillres = avpicture_fill((AVPicture *)tmp_picture, picture_buf, codecContext->pix_fmt, codecContext->width, codecContext->height);

	int out_size = 0;

		// color convert the image... swsConvert is a company wrapper for sws_scale
		bool success = swsConvert(tmp_picture, rgbimage, codecContext->pix_fmt);
		tmp_picture->quality = FF_QP2LAMBDA;

		// encode the stream into video_outbuf
		// returns a negative value on error
		out_size = avcodec_encode_video(codecContext, video_outbuf, video_outbuf_size, tmp_picture);
		
		mFrameQueue->pop_front();
		if (out_size > 0)
		{
			// create an AVPacket using the encoded imagery
			AVPacket pkt;
			av_init_packet(&pkt);

			pkt.data = video_outbuf;
            pkt.size = out_size;
			pkt.stream_index = vstream->index;
            pkt.pts= av_rescale_q(codecContext->coded_frame->pts, codecContext->time_base, vstream->time_base);
            pkt.flags |= AV_PKT_FLAG_KEY;

			// write the frame
            if (av__write_frame(muxer, &pkt) < 0)
            {
                   // error handling
            }

			av_free_packet(&pkt);

	av_free(tmp_picture);
	delete[] picture_buf;
}