[FFmpeg-devel] EXT-X-PROGRAM-DATE-TIME value drifts behind the actual time on the start of the corresponding segment video

[Vignesh Ravichandran]

Friendly ping. Would greatly appreciate some input on this.

Thanks,
Vignesh

On Fri, Jan 22, 2021 at 3:35 PM Vignesh Ravichandran <
vignesh.ravichandran02 at gmail.com> wrote:

> I am trying to create a video by stitching multiple HLS segments  (.ts
> files)  together based on a start time and end time.  Basically, create a
> video of a particular timeline in the HLS stream.
>
> My setup has a camera pointing to a monitor that displays the system clock
> (The camera control and the ffmpeg command run, is on the same system).
> The frame rate of the camera is not constant but is capped at 25 FPS. The
> frames captured from the camera are passed on to the following ffmpeg
> command:
> ```
> /usr/bin/ffmpeg -vaapi_device /dev/dri/renderD128 -y -v 9 -loglevel 99 -f
> rawvideo -vcodec rawvideo -s 1920x1200 -pix_fmt uyvy422 -i - -an -r 25
> -vcodec h264_vaapi -vf format=nv12,hwupload -force_key_frames
> expr:gte(t,n_forced*1) -f hls -hls_list_size 10080 -hls_time 10 -hls_flags
> delete_segments -hls_flags second_level_segment_index -hls_flags
> +append_list+discont_start+program_date_time -strftime 1 -strftime_mkdir 1
> -hls_segment_filename %Y/%m/%d/%%04d.ts playlist.m3u8
> ```
>
> I parse the playlist file created and then pick up a set of .ts files that
> lie between the given start and end time. This is done by comparing the
> “#EXT-X-PROGRAM-DATE-TIME” value of each file in the playlist. Then, I
> stitch these files together and trim to produce the final output video file.
>
> Problem Encountered:
> I noticed that the “#EXT-X-PROGRAM-DATE-TIME” value in the playlist file
> for most segments and the system time shown in the start (00:00:00) of the
> corresponding segment video is different. The “#EXT-X-PROGRAM-DATE-TIME” is
> behind. For example, for below entry:
> ```
> #EXTINF:10.000000,
> #EXT-X-PROGRAM-DATE-TIME:2021-01-21T23:17:05.000+1100
> 2021/01/21/0085.ts
> ```
> the value shown on the system clock in the video is
> "2021-01-21T23:18:12.000+1100" which is more than a minute ahead.
> This causes the final video created to contain incorrect timeline from the
> stream.
>
> My observation and conclusion:
> Since the FPS of the camera is not constant and can vary between 0 to 25,
> ffmpeg would have to duplicate frames when the input FPS drops below 25.
> But I don’t see any mention of this happening in the logs. So I think
> ffmpeg is filling in the current segment with frames outside the segment
> duration i.e., with frames which are supposed to be in the next segment.
> And the experiment mentioned above verifies it.
>
> Looking at the code, the “do_video_out” function in “ffmpeg.c” file has
> logic to duplicate frames if needed. But not sure why this is not
> happening. Would greatly appreciate some help in debugging and resolving
> this issue. I have also attached the log file from my test here.
>
> Thanks,
> Vignesh
>
>  ffmpeg.log
> <https://drive.google.com/file/d/1FDx2LzVNksl2w-6fz9ShC7vvbrhEXsQv/view?usp=drive_web>
>
>