[Libav-user] achieving greater than 15 fps playback

Blake Senftner bsenftner at earthlink.net
Sun Dec 18 20:43:55 EET 2016


And I’m on an i7 8 core desktop with 32 GB ram, Win10, deploying to Windows and Linux oses. 

Sincerely,
-Blake Senftner
Mad Computer Scientist

> On Dec 18, 2016, at 9:22 AM, Blake Senftner <bsenftner at earthlink.net> wrote:
> 
> Hello,
> 
> Working on video (no audio) playback library, I am only seeing frame rates of 15 fps or less. My code started based on the Dranger examples, but has since been updated to the latest APIs (such that I compile with no depreciated warnings.) I am using Visual Studio 2013 Community. 
> 
> I have a 160x40 29.97 fps h264 clip with timecodes, and it plays back at exactly half speed, consuming very little cpu. An rtsp h264 stream from an IP camera plays back between 10 & 11 fps, while the latest full HD resolution Rogue One trailer plays back at 15 fps (both only consuming < 20% CPU.) 
> 
> My code does not use SDL, so I am using neosmart cross platform pevents (https://github.com/neosmart/pevents). 
> 
> I am using one thread to read packets, decode them to yuv & convert to RGB. OpenGL does the display in the main thread.
> 
> Does achieving higher frame rates require the packet reading and packet decoding in separate threads? I thought the 160x40 file would decompress fast enough that I’d need to add necessary timing logic to delay the frames, but that is not the case. It plays
> 
> Here’s the general logic of my playback setup, abbreviated for clarity, and the packet reading is below that:
> 
> ------------------------------------------------------------------------------------------
> avformat_open_input( &mp_format_context, [path to file or stream], NULL, NULL );
> avformat_find_stream_info( mp_format_context, NULL );
> m_video_stream = -1;
> for (ce_uint i = 0; i < (ce_uint)mp_format_context->nb_streams; i++) {
> 	if (mp_format_context->streams[i]->codecpar->codec_type==AVMEDIA_TYPE_VIDEO) {
> 		m_video_stream = i;
> 		break;
> 	}
> }
> if (m_video_stream == -1) 
> 	return false;
> 
> AVCodecParameters* codecPars = mp_format_context->streams[m_video_stream]->codecpar;
> 
> AVCodec* pCodec = avcodec_find_decoder( codecPars->codec_id );
> if (!pCodec) 
> 	return false;
> 
> mp_codec_context = avcodec_alloc_context3( pCodec );
> if (avcodec_parameters_to_context( mp_codec_context, codecPars ) != 0)
> 	return false;
> 	
> mp_codec_context->thread_count = 0; // tried 4, 8… makes no difference
> 
> AVDictionary* codec_options = NULL;
> av_dict_set( &codec_options, "threads", "auto", 0 );	// makes no difference, tried 4 & 8 too
> if (m_stream_type == 2)
> 	av_dict_set( &codec_options, "rtsp_transport", "tcp", 0 ); // needed for ffplay to work with my ip cam, makes no difference here...
> 
> if (avcodec_open2( mp_codec_context, pCodec, &codec_options ) < 0)
> 	return false;
> 
> mp_decompressed_frame = av_frame_alloc();
> mp_display_frame = av_frame_alloc();
> if (mp_decompressed_frame == NULL || mp_display_frame == NULL)
> 	return false;
> 	
> ce_int numBytes = av_image_get_buffer_size( AV_PIX_FMT_RGBA, mp_codec_context->width, mp_codec_context->height, 1 );
> mp_display_buffer = (uint8_t *)av_malloc( numBytes*sizeof(uint8_t) );
> av_image_fill_arrays( mp_display_frame->data, mp_display_frame->linesize,		
> 		mp_display_buffer, AV_PIX_FMT_RGBA,
> 		mp_codec_context->width, mp_codec_context->height, 1 );
> 
> mp_sws_context = sws_getContext( mp_codec_context->width, mp_codec_context->height, mp_codec_context->pix_fmt,
> 																	 mp_codec_context->width, mp_codec_context->height, AV_PIX_FMT_RGBA,
> 																	 SWS_BILINEAR, NULL, NULL, NULL );
> 
> —————————————————————————————————————————
> 
> In my packet reading thread, the logic is simple:
> 
> ——————————————————————————————————
> uint64_t milliseconds = 1;
> 
> bool media_has_ended = false;
> bool camera_has_terminated = false;
> ce_uint packet_errors = 0;
> 
> while (true) {
> 	// this basically allows the thread to idle (sleep) but it will
> 	// wake up if the events are signaled
> 	dwWaitRes = neosmart::WaitForEvent(m_stop_local_spin_event, milliseconds);
> 
> 	if (0 == dwWaitRes) // stop event
> 		break;
> 		
> 	else if (m_is_playing && !m_paused) {
> 
> 		// I’ve got 16 packet buffers so I can look at their data, but only use one at a time: 
> 		ce_uint curr_packet_index = m_total_video_packets % CE_LIBAV_NUMVIDPKTS;
> 		AVPacket* curr_packet = &m_packet[ curr_packet_index ];
> 		m_total_video_packets++;
> 
> 		int stream_status = av_read_frame(mp_format_context, curr_packet);
> 		if (stream_status < 0) {
> 
> 			if (m_stream_type == 0) { // end of media file // stream_types: 0=Media, 1=USB, 2=IP
> 				media_has_ended = true;
> 				break;
> 			}
> 			else { // camera stream has terminated unexpectedly 
> 				camera_has_terminated = true;
> 				break;
> 			}
> 		}
> 		else { // stream delivered a packet fine 
> 			
> 			if (curr_packet->stream_index == m_video_stream) {
> 				
> 				// Decode video frame(s):
> 				int ret = avcodec_send_packet( mp_codec_context, curr_packet );
> 				if (ret < 0) {
> 					
> 					if (ret == AVERROR_EOF && m_stream_type == 0) {
> 						media_has_ended = true;
> 						break;
> 					}
> 					else packet_errors++;
> 				}
> 				else {
> 					
> 					while (!ret)
> 					{
> 						ret = avcodec_receive_frame( mp_codec_context, mp_decompressed_frame );
> 						if (!ret)
> 							handle_new_frame( curr_packet, curr_packet_index );
> 						
> 					}
> 	
> 				}
> 				av_packet_unref(curr_packet);
> 			
> 		}		// end stream delivered a packet
> 
> 	}			// end if (m_is_playing && !m_paused)
> }				// end while (true)
> 
> if (media_has_ended) {
> 	if (mp_stream_ended_callback) 
> 			(mp_stream_ended_callback)(m_frames_received, mp_stream_ended_object);
> 	
> if (camera_has_terminated)
> 	if (mp_term_callback) 
> 			(mp_term_callback)(mp_term_object);
> 	
> 
> ——————————————————————————————————
> 
> 
> Sincerely,
> -Blake Senftner
> Mad Computer Scientist
> 
> p.s.
> 
> My ffmpeg libav* 3.2 libs are, downloaded pre-built from Zeranoe early November, my versions are:
> 
> libavutil      55. 34.100 / 55. 34.100
> libavcodec     57. 64.100 / 57. 64.100
> libavformat    57. 56.100 / 57. 56.100
> libavdevice    57.  1.100 / 57.  1.100
> libavfilter     6. 65.100 /  6. 65.100
> libswscale      4.  2.100 /  4.  2.100
> libswresample   2.  3.100 /  2.  3.100
> libpostproc    54.  1.100 / 54.  1.100
> 
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user



More information about the Libav-user mailing list