[FFmpeg-user] Unable to record VP8+Opus from RTP+SDP
Juan Navarro
juan.navarro at gmx.es
Wed Aug 14 17:06:27 EEST 2019
>> So I’m not sure how ffmpeg parses SDP files or if it uses a library
but when you step back and take a look, you haven’t put in any specifics
for either codec besides from maybe their names.
>>
>> v=0
>> o=- 0 0 IN IP4 127.0.0.1
>> s=-
>> c=IN IP4 127.0.0.1
>> t=0 0
>> m=audio 5006 RTP/AVP 109
>> a=rtpmap:109 opus/48000/2
>> m=video 5004 RTP/AVP 120
>> a=rtpmap:120 VP8/90000
>
> You can try reading the rfc’s and filling in the extra codec info it
needs, but if you’re the one setting up the session, why not just use a
simpler protocol? (I’m assuming the localhost is just a stand-in)
>
>> However, the issue persists when trying to record Opus audio; it is just
>> not possible to play it back, supposedly due to missing headers (but
>> then how is it possible to record incoming RTP Opus audio?)
>
> Like, you told ffmpeg it was going to be 48khz, 2 chs it believed
your copy codec and just dumped everything into a file. But when you
open up the file, does it really have two channels? is the audio
interleaved? Nobody knows.
>
I just want to have a working command to record RTP media received from
an RTP router, in this case it is MediaSoup, which receives a WebRTC
stream from Chrome, then redistributes it as plain RTP. I first tried
using plain "rtp://" URL, but the ffmpeg command itself told me that an
SDP input file would be needed to use dynamic Payload Types (which is
totally sensible).
This is a standard SDP Offer as generated by Chrome to create a WebRTC
session:
v=0
o=- 6323209318560568134 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=msid-semantic: WMS XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Ao6s
a=ice-pwd:h3NoIquJSZ8L/yQUSy7nme5Y
a=ice-options:trickle
a=fingerprint:sha-256
21:D2:14:1E:49:95:70:0B:A6:0B:93:5C:E5:A9:35:B0:8A:48:3C:95:75:06:62:74:5A:2B:99:29:0F:D0:3E:FB
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2
http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
816c42db-7d97-4053-86a0-bae0e5010985
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3169356447 cname:XZAw22pKY7Y9bCph
a=ssrc:3169356447 msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
816c42db-7d97-4053-86a0-bae0e5010985
a=ssrc:3169356447 mslabel:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
a=ssrc:3169356447 label:816c42db-7d97-4053-86a0-bae0e5010985
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 122 127 121 125
107 108 109 124 120 123
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Ao6s
a=ice-pwd:h3NoIquJSZ8L/yQUSy7nme5Y
a=ice-options:trickle
a=fingerprint:sha-256
21:D2:14:1E:49:95:70:0B:A6:0B:93:5C:E5:A9:35:B0:8A:48:3C:95:75:06:62:74:5A:2B:99:29:0F:D0:3E:FB
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:12 urn:3gpp:video-orientation
a=extmap:2
http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6
http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
0c05ddde-b52e-4776-97f0-755ded96e15d
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102
level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127
level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:121 rtx/90000
a=fmtp:121 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125
level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108
level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:124 red/90000
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=124
a=rtpmap:123 ulpfec/90000
a=ssrc-group:FID 3597912382 1560435023
a=ssrc:3597912382 cname:XZAw22pKY7Y9bCph
a=ssrc:3597912382 msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
0c05ddde-b52e-4776-97f0-755ded96e15d
a=ssrc:3597912382 mslabel:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
a=ssrc:3597912382 label:0c05ddde-b52e-4776-97f0-755ded96e15d
a=ssrc:1560435023 cname:XZAw22pKY7Y9bCph
a=ssrc:1560435023 msid:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
0c05ddde-b52e-4776-97f0-755ded96e15d
a=ssrc:1560435023 mslabel:XAb2pgeSAM6YZw6uCRj11AE5Raf3cKHPNGl0
a=ssrc:1560435023 label:0c05ddde-b52e-4776-97f0-755ded96e15d
If you notice, there is a lot of metadata about the RTP session (ssrc,
cname), some about the ICE protocol (for WebRTC), but there isn't much
information there about the specifics of the codecs. In this case, the
preferred audio payload type is 111, which corresponds to the OPUS
codec. The specification "opus/48000/2" is expected to already convey
enough information to let the other peer know what codec it is going to
receive, and it allows for some (much) variance in the actual encoding
parameters that are going to be used; the receiver is just able to
auto-detect those and decode the stream.
For the cases where the codec has really incompatible profiles, to the
point where trying to decode with the wrong profile would fail, then
that information is also transmitted in the SDP; this is the case of the
H.264 profile levels (Baseline, Main, High, etc.), which are then
accordingly transmitted in the SDP Offer as totally different Payload
Types. But you'll notice that other codecs, like VP8, don't really need
such distinction.
Of course, such an SDP Offer is not really missing any crucial detail;
another browser such as Chrome, Firefox, Safari, etc. will be able to
receive and decode the incoming audio and video. So what I'm really
trying to do is doing the same with FFmpeg... now if you compare
Chrome's SDP with mine, it turns out mine does already have all relevant
details that are included in a "full-blown" SDP message.
More information about the ffmpeg-user
mailing list