[FFmpeg-trac] #3586(avcodec:new): [discrepancy with libav] avcodec_encode_video2 merges packet side data with the packet body

[FFmpeg]

#3586: [discrepancy with libav] avcodec_encode_video2 merges packet side data with
the packet body
-------------------------------------+-------------------------------------
             Reporter:  Lastique     |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:  avcodec      |                  Version:  2.2.1
             Keywords:  libav        |               Blocked By:
  avcodec_encode_video2 avpacket     |  Reproduced by developer:  0
  av_packet_merge_side_data          |
             Blocking:               |
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 We have an application which uses ffmpeg to encode video in h263. The
 application requests h263_mb_info side data by setting the "mb_info"
 option on the encoder context:

 {{{
 av_opt_set_int(av_ctx, "mb_info", 1400, AV_OPT_SEARCH_CHILDREN)
 }}}

 Later we call avcodec_encode_video2() to encode frames. The problem is
 that ffmpeg avcodec_encode_video2() returns AVPackets with the encoded
 frame and some amount of junk at the end. From the source code I can see
 that the junk is actually the encoded side data (mb_info) appended to the
 h263 packet by av_packet_merge_side_data() which is called by
 avcodec_encode_video2().

 This appended side data prevents the packets from being decoded by ffmpeg
 2.2.1 (I tried both decoding through the API and ffplay). Decoding results
 in many messages such as:

 {{{
 slice end not reached but screenspace end (399 left 019CE3, score= -7)
 concealing 88 DC, 88 AC, 88 MV errors in P frame
 }}}

 and corrupted image.

 This is also different from libav behavior. At least 9.x and 10.x versions
 don't merge side data into the packet body, and when using libav we don't
 have the decoding problem.

 Looking at the libavcodec code I don't really understand why the call to
 av_packet_merge_side_data() is needed. Side data, at least mb_info in case
 of h263, is not the part of the encoded media and thus should not be part
 of the packet body. Since the packet size (AVPacket::size) includes the
 appended side data, there is no way to obtain the size of the actual
 media. I suspect, av_packet_split_side_data() can be used to restore the
 original un-merged packet, but (a) it is not documented and (b) doing
 av_packet_merge_side_data() and av_packet_split_side_data() is a waste of
 memory and CPU (note: we pre-allocate packet data buffer to optimize
 performance). Therefore my preferred solution would be to remove the call
 to av_packet_merge_side_data() from avcodec_encode_video2().

 If that call cannot be removed then what is the correct way to obtain the
 actual media from the merged packet?

--
Ticket URL: <https://trac.ffmpeg.org/ticket/3586>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker