[FFmpeg-devel] [PATCH 1/2] avcodec/parser: fix fetch_timestamp in a scenario with unaligned packets

[Nicolas Gaullier]

>De : ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> De la part de Michael Niedermayer
>Envoyé : mercredi 21 février 2024 05:32
>On Tue, Feb 20, 2024 at 05:33:01PM +0100, Nicolas Gaullier wrote:
>> Fix fetch_timestamp when the frame start is in a previous packet.
>> 
>> Signed-off-by: Nicolas Gaullier <nicolas.gaullier at cji.paris>
>> ---
>>  libavcodec/parser.c | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>
>This change looses pts

I missed it : some broken streams are missing the zero_byte which makes the current h264 parser code to borrow a terminating null byte in the previous frame if available.
It seems there is currently no issue with that behaviour, but with my patch fixing the fetch_timestamp mechanism, it becomes one.
So, what is somewhat tricky is to guess if we are facing a broken stream or a conformant stream which actually has its zero_byte in the previous frame.
In my experience (including the sample from Michael and a sample of mine where there is no available null byte at the end of the frame),
the "usual broken streams" are missing the zero_byte for the first NAL unit which is an AUD, but the following NAL still has this zero_byte.
The following patch is a proposal to detect and overcome such a situation:
https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=10877
At the end, this patch is required to handle broken streams and thus prepare the ground for the fetch_timestamp patch.

Another option would be for example to handle the data_alignment_indicator in the mpegts demuxer to force the alignment (ex: with a parser state reset),
but it seems it would involve some big unhappy changes in the code, with demux and parser tied together. Moreover, I don't think it is reliable and there might exists
broken stream with unaligned packets that we would still like to support.

Any inputs concerning broken streams for other codecs is welcome. For example, it may be required to handle broken hevc streams alike h264 ones: I have no opinion/ samples for that matter.

Nicolas