Expected latency with RTSP/TCP H264 stream
Hello, I'm trying to have a low latency stream if possible, less than 500ms. By "latency" I mean the delta between a frame is sent by the camera and when it's received and decoded by FFMpeg until being available to the stdout. To measure it, I add an overlay from the camera side with the source timestamp And I then add a timestamp on the ffmpeg side using the drawtext plugin and output it as a BMP. My measurement show: 1080p: 616ms 4k: 699ms I use the Cuvid decoder with the following command: ./ffmpeg -allowed_media_types video -c:v h264_cuvid -err_detect careful -fflags nobuffer -init_hw_device cuda:0 -max_port 65000 -min_port 5000 -nostats -re -rtsp_flags prefer_tcp -rtsp_transport tcp -surfaces 8 -timeout 1000000 -i rtsp://root:XXX@10.98.5.107:554/axis-media/media.amp -vf "drawtext=text='%{localtime\:%Y/%m/%d %H\\\\\:%M\\\\\:%S.%N}': x=0: y=50: fontcolor=white: fontsize=48: box=1: boxcolor=black@0.5" -pix_fmt rgb24 -vframes 1 output_image_X.bmp Do you think reducing the latency to less than 500ms is doable? Or is there some kind of limitation in the pipeline somewhere? Thanks for your help
Hello, Replying to myself, I hope this can help other people. Easier for debugging use -debug_ts and look at the line: muxer <- pts:18 pts_time:2 dts:18 dts_time:2 duration:1 duration_time:0.111111 size:24883254 latency(total:547.743ms, demux-decode: 133.354ms/24%, decode 403.334ms/73%) This will give out the ffmpeg latency (avoid starting and flushing packets). Which is what I measure myself but easier. And after more research in the libavcodec/cuviddec.c I found there is this tweak https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/cuviddec.c#L1049 Adding the "-flags +low_delay" will give you muxer <- pts:47 pts_time:5.222222 dts:47 dts_time:5.222222 duration:1 duration_time:0.111111 size:24883254 latency(total:104.805ms, decode 96.373ms/91%) So 105ms, which is quite nice :) If anyone as another trick, please let me know Thanks On Fri, 1 Nov 2024 at 18:06, Clément Péron <peron.clem@gmail.com> wrote:
Hello,
I'm trying to have a low latency stream if possible, less than 500ms. By "latency" I mean the delta between a frame is sent by the camera and when it's received and decoded by FFMpeg until being available to the stdout.
To measure it, I add an overlay from the camera side with the source timestamp And I then add a timestamp on the ffmpeg side using the drawtext plugin and output it as a BMP.
My measurement show: 1080p: 616ms 4k: 699ms
I use the Cuvid decoder with the following command: ./ffmpeg -allowed_media_types video -c:v h264_cuvid -err_detect careful -fflags nobuffer -init_hw_device cuda:0 -max_port 65000 -min_port 5000 -nostats -re -rtsp_flags prefer_tcp -rtsp_transport tcp -surfaces 8 -timeout 1000000 -i rtsp://root:XXX@10.98.5.107:554/axis-media/media.amp -vf "drawtext=text='%{localtime\:%Y/%m/%d %H\\\\\:%M\\\\\:%S.%N}': x=0: y=50: fontcolor=white: fontsize=48: box=1: boxcolor=black@0.5" -pix_fmt rgb24 -vframes 1 output_image_X.bmp
Do you think reducing the latency to less than 500ms is doable? Or is there some kind of limitation in the pipeline somewhere?
Thanks for your help
Clément Péron wrote:
Hello,
Replying to myself, I hope this can help other people.
Pleas avoid top-posting.
[...] Adding the "-flags +low_delay" will give you muxer <- pts:47 pts_time:5.222222 dts:47 dts_time:5.222222 duration:1 duration_time:0.111111 size:24883254 latency(total:104.805ms, decode 96.373ms/91%)
You might also try "-tune zerolatency". Best regards -- Oliver
Hi Olivier On Sun, 3 Nov 2024 at 14:32, Oliver Fromme <oliver@fromme.com> wrote:
Clément Péron wrote:
Hello,
Replying to myself, I hope this can help other people.
Pleas avoid top-posting.
Sorry for this inconvenience, I realized it but it was already too late...
[...] Adding the "-flags +low_delay" will give you muxer <- pts:47 pts_time:5.222222 dts:47 dts_time:5.222222 duration:1 duration_time:0.111111 size:24883254 latency(total:104.805ms, decode 96.373ms/91%)
You might also try "-tune zerolatency".
[in#0/rtsp @ 0x556fb8629740] Codec AVOption tune (Set the encoding tuning info) is not a decoding option. Error opening input file rtsp://root:XXX@10.98.5.107:554/axis-media/media.amp. Seems this is only for the nvenc, but thanks for your suggestion Regads, Clément
Best regards -- Oliver _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
Clément Péron wrote:
On Sun, 3 Nov 2024 at 14:32, Oliver Fromme <oliver@fromme.com> wrote:
Clément Péron wrote:
[...] Adding the "-flags +low_delay" will give you muxer <- pts:47 pts_time:5.222222 dts:47 dts_time:5.222222 duration:1 duration_time:0.111111 size:24883254 latency(total:104.805ms, decode 96.373ms/91%)
You might also try "-tune zerolatency".
[in#0/rtsp @ 0x556fb8629740] Codec AVOption tune (Set the encoding tuning info) is not a decoding option. Error opening input file rtsp://root:XXX@10.98.5.107:554/axis-media/media.amp.
Seems this is only for the nvenc, but thanks for your suggestion
Maybe I've been mistaken, but from you command line it appeared to me that you were using libx264 for encoding. In that case, "-tune zerolatency" is a short-cut for several encoding options. Among other things, it disables B-frames and various other algorithms that require looking ahead at future frames, and it enables sliced threading (which is less efficient than the default of frame-based threading, but causes less delay). If you don't use libx264 for encoding, then ignore what I wrote. :-) Best regards -- Oliver
On Sun, 3 Nov 2024 at 21:32, Oliver Fromme <oliver@fromme.com> wrote:
Clément Péron wrote:
On Sun, 3 Nov 2024 at 14:32, Oliver Fromme <oliver@fromme.com> wrote:
Clément Péron wrote:
[...] Adding the "-flags +low_delay" will give you muxer <- pts:47 pts_time:5.222222 dts:47 dts_time:5.222222 duration:1 duration_time:0.111111 size:24883254 latency(total:104.805ms, decode 96.373ms/91%)
You might also try "-tune zerolatency".
[in#0/rtsp @ 0x556fb8629740] Codec AVOption tune (Set the encoding tuning info) is not a decoding option. Error opening input file rtsp://root:XXX@10.98.5.107:554/axis-media/media.amp.
Seems this is only for the nvenc, but thanks for your suggestion
Maybe I've been mistaken, but from you command line it appeared to me that you were using libx264 for encoding. In that case, "-tune zerolatency" is a short-cut for several encoding options. Among other things, it disables B-frames and various other algorithms that require looking ahead at future frames, and it enables sliced threading (which is less efficient than the default of frame-based threading, but causes less delay).
If you don't use libx264 for encoding, then ignore what I wrote. :-)
Sorry my first message was imprecise, I got the H264 RTSP stream from a camera. I'm only interested in decoding video, frame by frame, as quickly as possible. My final result is a total E2E of 300-450ms where between 50-200ms are from FFMpeg decoding. Strange is that the Cuvid decoding has a high jitter from 40ms to 190ms. Maybe I will investigate this part. Regards, Clement
Best regards -- Oliver _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user
To unsubscribe, visit link above, or email ffmpeg-user-request@ffmpeg.org with subject "unsubscribe".
participants (2)
-
Clément Péron -
Oliver Fromme