[FFmpeg-trac] #8466(avformat:new): av_url_split violates RFC2396, fails to parse URLs
FFmpeg
trac at avcodec.org
Fri Jan 10 09:04:45 EET 2020
#8466: av_url_split violates RFC2396, fails to parse URLs
-------------------------------------+-------------------------------------
Reporter: aitte | Type: defect
Status: new | Priority: important
Component: avformat | Version: 4.2
Keywords: RFC2396 | Blocked By:
http parsing |
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
''(Slightly related ticket: #7871)''
'''Summary of the bug:'''
This took a long time to track down, but the exact file and problem has
now been identified.
The bug is in "libavformat/utils.c", function "av_url_split".
When given a URL that conforms to RFC2398, the 1998 URL standard, FFmpeg
incorrectly parses the URL and fails to download the file.
This is because FFmpeg's parser looks for the earliest slash after the
domain name, by assuming that there must be a slash after the domain name,
before the query string, which is not true. The standard says that the
slash is totally optional (https://stackoverflow.com/a/42193734/8874388).
'''How to reproduce:'''
{{{
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.1.1 (GCC) 20190807
INCORRECTLY PARSED BY FFMPEG:
% ffmpeg -i "https://key.vscdns.com?key_id=2&model_id=991011&&access_key="
[https @ 000002308f499b00] HTTP error 400 Bad Request
CORRECTLY PARSED (VIA USER MANUALLY ADDING A SLASH):
% ffmpeg -i
"https://key.vscdns.com/?key_id=2&model_id=991011&&access_key="
[https @ 00000262520c9b00] HTTP error 403 Forbidden
}}}
It affects any URL where the query string is immediate. Which means that
it affects direct media links, M3U8/HLS playlists (where the user has no
control over the URLs and no way to fix it themselves), etc.
This is problematic because the ".com?" format is entirely up to the
streaming media provider, and cannot be affected by the user. Any
streaming playlist that uses a field such as "EXT-X-KEY" with such a URL
(as in my example URL) will fail completely in FFmpeg and there's nothing
the user can do about it. So it is imperative that FFmpeg understands
RFC2396 and allows the slash to be optional.
I looked at the parsing code and it seems quite simple to fix. It
currently looks for the first "/" slash and "?" question mark. The fix
would probably be to say "If a question mark was found, and it is earlier
than the first slash, or no slash was found at all" then set the pointers
so that it treats the "?" as the start of the query and the character
''immediately before that'' as the end of the domain name.
PS: I have set the same priority as the earlier URL parsing error ticket I
referred to above. Feel free to lower the priority.
--
Ticket URL: <https://trac.ffmpeg.org/ticket/8466>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list