[FFmpeg-user] Audio decode adding 100ms delay

[Max Rowan]

Hi all and thanks in advance for your responses. I've run into an issue where decoding audio adds roughly 100ms of delay to the entire decode. Simply adding the "-an" flag to either the encode/decode immediately takes 100ms off the overall latency. For reference I'm doing a capture from a decklink card and hardware encoding the video out to a UDP stream, which is then wrapped in an SRT stream. I'm not sure if it is of any consequence, but with "-loglevel debug" enabled the stream output for the opus encoder states "delay 120", like so:

Stream #0:1, 0, 1/90000: Audio: opus (libopus), 48000 Hz, stereo, s16, delay 120, 128 kb/s

 I can't find where this "delay" is coming from, how it is calculated, and any change I make to the setting of libupos can never reduce this "delay" below 120. I should say that I had the same issue with other audio codecs as well. 

Does anyone have any idea where this 100ms is coming from?

Encoding:

sudo ffmpeg -f decklink -duplex_mode half -format_code Hp50 -queue_size 4976640 -i "DeckLink Duo (1)" -c:v h264_qsv -b:v 6M -maxrate 6M -preset veryslow -bf 0 -refs 1 -c:a libopus -b:a 128k -compression_level 0 -frame_duration 20 -application lowdelay -f mpegts  "udp://127.0.0.1:1234?pkt_size=1316"

SRT Wrap:

srt-live-transmit udp://:1234 "srt://10.0.3.5:1235?mode=listener&pkt_size=1316&latency=0"

Decode with audio:

ffplay -fast -fflags nobuffer -flags low_delay -framedrop -strict experimental "srt://10.0.3.5:1235?mode=caller&latency=0"

Decode no audio:

ffplay -fast -fflags nobuffer -flags low_delay -framedrop -strict experimental -an "srt://10.0.3.5:1235?mode=caller&latency=0"

Encode Output:

sudo ffmpeg -f decklink -duplex_mode half -format_code Hp50 -queue_size 4976640 -i "DeckLink Duo (1)" -c:v h264_qsv -b:v 6M -maxrate 6M -preset veryslow -bf 0 -refs 1 -c:a libopus -b:a 128k -compression_level 0 -frame_duration 20 -application lowdelay -f mpegts  "udp://127.0.0.1:1234?pkt_size=1316"
ffmpeg version N-98687-g6e951d0cf8 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/root/ffmpeg_build --pkg-config-flags=--static --extra-cflags='-I/root/ffmpeg_build/include -I/home/max/ffmpeg_sources/BMD_SDK/include' --extra-ldflags='-L/root/ffmpeg_build/lib -L$/home/max/ffmpeg_sources/BMD_SDK/include' --extra-libs='-lpthread -lm' --bindir=/root/bin --enable-libmfx --enable-gpl --enable-gnutls --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libsrt --enable-decklink --enable-libvmaf --enable-nonfree
  libavutil      56. 58.100 / 56. 58.100
  libavcodec     58.100.100 / 58.100.100
  libavformat    58. 50.100 / 58. 50.100
  libavdevice    58. 11.101 / 58. 11.101
  libavfilter     7. 87.100 /  7. 87.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
[decklink @ 0x55b52ce07140] Found Decklink mode 1920 x 1080 with rate 50.00
[decklink @ 0x55b52ce07140] Frame received (#1) - No input signal detected - Frames dropped 1
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, decklink, from 'DeckLink Duo (1)':
  Duration: N/A, start: 0.000000, bitrate: 1660416 kb/s
    Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
    Stream #0:1: Video: rawvideo (UYVY / 0x59565955), uyvy422(progressive), 1920x1080, 1658880 kb/s, 50 tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:1 -> #0:0 (rawvideo (native) -> h264 (h264_qsv))
  Stream #0:0 -> #0:1 (pcm_s16le (native) -> opus (libopus))
Press [q] to stop, [?] for help
Output #0, mpegts, to 'udp://127.0.0.1:1234?pkt_size=1316':
  Metadata:
    encoder         : Lavf58.50.100
    Stream #0:0: Video: h264 (h264_qsv), nv12(progressive), 1920x1080, q=-1--1, 6000 kb/s, 50 fps, 90k tbn, 50 tbc
    Metadata:
      encoder         : Lavc58.100.100 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 6000000/0/6000000 buffer size: 0 vbv_delay: N/A
    Stream #0:1: Audio: opus (libopus), 48000 Hz, stereo, s16, 128 kb/s
    Metadata:
      encoder         : Lavc58.100.100 libopus

Thanks in advance, Max