[Ffmpeg-devel] Trouble decoding Live RTP H.264 stream

[Kenyon, Sam]

Summary: I am trying to sink H264 video data from an RTSP/RTP stream via
LiveMedia to avcodec for decoding.  But the avcodec_decode_video
function never sets got_picture_pointer to non-zero, and the frame
structure never gets filled with decoded data.

This exact same stream can be displayed by VLC, which says in its
message log that it is using Live_dot_com and ffmpeg.

libavcodec version: 51.7.0.

This is my decode wrapper function:

//frameFinished 
bool FfMpegDecoder::decodeFrame(uint8_t *rawData, unsigned int
data_size)
{
    static int frameFinished = 0;  //this is the got_picture_pointer
argument
    int bytesDecoded=0;
    int bytesRemaining = data_size;

    while(bytesRemaining > 0)
    {
        bytesDecoded=avcodec_decode_video(pCodecCtx, pFrame,
            &frameFinished, rawData, bytesRemaining);

        if(bytesDecoded < 0)
            return false;

        bytesRemaining-=bytesDecoded;
        rawData+=bytesDecoded;

        if(frameFinished)
            return true;
    }

    // Decode the rest of the last frame
    bytesDecoded=avcodec_decode_video(pCodecCtx, pFrame, &frameFinished,

        rawData, bytesRemaining);

    return frameFinished!=0;
}


I have some ideas of what the problem might be, but I haven't made much
progress--perhaps someone here can point out my mistake or relate a
similar experience.

Possibility 1:
I am missing header or meta data that avcodec's h264 decoder needs,
maybe because of the RTP payload differences.

The first choking point may be that the extradata_size is zero, although
I tried a hack with extradata set from avformat (looking at a separate
h264 file) and it still didn't get any pictures.

The second choking point is that it appears to never find the frame
start or the frame end.

When comparing the packets from av_read_packet() (when loading an h.264
file (which works fine)) to the packets coming from LiveMedia, I noticed
two things:


* the LiveMedia chunks always start with 0x21, which I assume is the
same 0x21 H.264 Visual stream object ID code.  But the av_read_packet
chunks (loaded from an h.264 file) don't start with 0x21, and the first
appearance of 0x21 is not constant.


* av_read_packet chunks are always 1024 bytes, but the chunks from
LiveMedia (this is in the addData function of a custom sink) are
variable.  The byte sizes are about what I'd expect based on the VBR
settings on my encoder.  As one would expect, every 30th frame is much
larger, since my encoder's I frame period is set to 30.



Possibility 2: My AVCodecContext  or usage is incorrect.  

However, I can decode and display data from an H264 file just fine.  For
the file I was using avformat to do most of the work for me.  I am
setting up the codec context manually for the LiveMedia sink, but
there's only a few fields to set for decoding that I can tell. I tried
playing with them, but still no got_picture_pointer.   I diffed the gdb
printouts of the codec context values (between the RTP streamed and the
file), and the only significant difference was the extradata (see the
paste below).  However, I did a test to use the context from the
avformat file loading (so it then had some extradata from the file), but
that didn't work either. 

This is a capture of the AVCodecContext object I am using to try and
decode h.264 data from a LiveMedia sink:

(gdb) print this.pCodecCtx 
$3 = (AVCodecContext *) 0x8535f30
(gdb) print *this.pCodecCtx 
$4 = {av_class = 0x8493be8
 bit_rate = 800000
 bit_rate_tolerance = 8000000
 flags = 65536
 sub_id = 0
 me_method = 5
 extradata = 0x0
 extradata_size = 0
 time_base = {num = 0
 den = 1}
 width = 352
 height = 240
 gop_size = 50
 pix_fmt = PIX_FMT_YUV420P
 rate_emu = 0
 draw_horiz_band = 0
 sample_rate = 0
 channels = 0
 sample_fmt = SAMPLE_FMT_S16
 frame_size = 0
 frame_number = 0
 real_pict_num = 0
 delay = 0
 qcompress = 0.5
 qblur = 0
 qmin = 2
 qmax = 31
 max_qdiff = 3
 max_b_frames = 0
 b_quant_factor = 1.25
 rc_strategy = 0
 b_frame_strategy = 0
 hurry_up = 0
 codec = 0x849bd00
 priv_data = 0xb3898020
 rtp_mode = 0
 rtp_payload_size = 0
 rtp_callback = 0
 mv_bits = 0
 header_bits = 0
 i_tex_bits = 0
 p_tex_bits = 0
 i_count = 0
 p_count = 0
 skip_count = 0
 misc_bits = 0
 frame_bits = 0
 opaque = 0x0
 codec_name = '\0' <repeats 31 times>
 codec_type = CODEC_TYPE_VIDEO
 codec_id = CODEC_ID_H264
 codec_tag = 0
 workaround_bugs = 1
 luma_elim_threshold = 0
 chroma_elim_threshold = 0
 strict_std_complian
ce = 0
 b_quant_offset = 1.25
 error_resilience = 1
 get_buffer = 0x8185310 <avcodec_default_get_buffer>
 release_buffer = 0x81840b0 <avcodec_default_release_buffer>
 has_b_frames = 0
 block_align = 0
 parse_only = 0
 mpeg_quant = 0
 stats_out = 0x0
 stats_in = 0x0
 rc_qsquish = 0
 rc_qmod_amp = 0
 rc_qmod_freq = 0
 rc_override = 0x0
 rc_override_count = 0
 rc_eq = 0x83fee8c "tex^qComp"
 rc_max_rate = 0
 rc_min_rate = 0
 rc_buffer_size = 0
 rc_buffer_aggressivity = 0
 i_quant_factor = -0.800000012
 i_quant_offset = 0
 rc_initial_cplx = 0
 dct_algo = 0
 lumi_masking = 0
 temporal_cplx_masking = 0
 spatial_cplx_masking = 0
 p_masking = 0
 dark_masking = 0
 unused = 0
 idct_algo = 0
 slice_count = 0
 slice_offset = 0x0
 error_concealment = 3
 dsp_mask = 0
 bits_per_sample = 0
 prediction_method = 0
 sample_aspect_ratio = {num = 0
 den = 1}
 coded_frame = 0x0
 debug = 0
 debug_mv = 0
 error = {0
 0
 0
 0}
 mb_qmin = 0
 mb_qmax = 0
 me_cmp = 0
 me_sub_cmp = 0
 mb_cmp = 0
 ildct_cmp = 8
 dia_size = 0
 last_predicto
r_count = 0
 pre_me = 0
 me_pre_cmp = 0
 pre_dia_size = 0
 me_subpel_quality = 8
 get_format = 0x81841d0 <avcodec_default_get_format>
 dtg_active_format = 0
 me_range = 0
 intra_quant_bias = 999999
 inter_quant_bias = 999999
 color_table_id = 0
 internal_buffer_count = 0
 internal_buffer = 0x0
 global_quality = 0
 coder_type = 0
 context_model = 0
 slice_flags = 0
 xvmc_acceleration = 0
 mb_decision = 0
 intra_matrix = 0x0
 inter_matrix = 0x0
 stream_codec_tag = 0
 scenechange_threshold = 0
 lmin = 236
 lmax = 3658
 palctrl = 0x0
 noise_reduction = 0
 reget_buffer = 0x8185220 <avcodec_default_reget_buffer>
 rc_initial_buffer_occupancy = 0
 inter_threshold = 0
 flags2 = 256
 error_rate = 0
 antialias_algo = 0
 quantizer_noise_shaping = 0
 thread_count = 1
 execute = 0x8184160 <avcodec_default_execute>
 thread_opaque = 0x0
 me_threshold = 0
 mb_threshold = 0
 intra_dc_precision = 0
 nsse_weight = 8
 skip_top = 0
 skip_bottom = 0
 profile = -99
 level = -99
 lowres = 0
 coded_width = 352
 coded_height = 240
 fra
me_skip_threshold = 0
 frame_skip_factor = 0
 frame_skip_exp = 0
 frame_skip_cmp = 13
 border_masking = 0
 mb_lmin = 236
 mb_lmax = 3658
 me_penalty_compensation = 256
 skip_loop_filter = AVDISCARD_DEFAULT
 skip_idct = AVDISCARD_DEFAULT
 skip_frame = AVDISCARD_DEFAULT
 bidir_refine = 0
 brd_scale = 0
 crf = 0
 cqp = -1
 keyint_min = 25
 refs = 1
 chromaoffset = 0
 bframebias = 0
 trellis = 0
 complexityblur = 20
 deblockalpha = 0
 deblockbeta = 0
 partitions = 0
 directpred = 2
 cutoff = 0
 scenechange_factor = 0}



Regards,
S.