[FFmpeg-trac] #8466(avformat:new): av_url_split violates RFC2396, fails to parse URLs

FFmpeg trac at avcodec.org
Fri Jan 10 09:04:45 EET 2020


#8466: av_url_split violates RFC2396, fails to parse URLs
-------------------------------------+-------------------------------------
             Reporter:  aitte        |                     Type:  defect
               Status:  new          |                 Priority:  important
            Component:  avformat     |                  Version:  4.2
             Keywords:  RFC2396      |               Blocked By:
  http parsing                       |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 ''(Slightly related ticket: #7871)''

 '''Summary of the bug:'''

 This took a long time to track down, but the exact file and problem has
 now been identified.

 The bug is in "libavformat/utils.c", function "av_url_split".

 When given a URL that conforms to RFC2398, the 1998 URL standard, FFmpeg
 incorrectly parses the URL and fails to download the file.

 This is because FFmpeg's parser looks for the earliest slash after the
 domain name, by assuming that there must be a slash after the domain name,
 before the query string, which is not true. The standard says that the
 slash is totally optional (https://stackoverflow.com/a/42193734/8874388).

 '''How to reproduce:'''

 {{{
 ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
   built with gcc 9.1.1 (GCC) 20190807

 INCORRECTLY PARSED BY FFMPEG:

 % ffmpeg -i "https://key.vscdns.com?key_id=2&model_id=991011&&access_key="
 [https @ 000002308f499b00] HTTP error 400 Bad Request

 CORRECTLY PARSED (VIA USER MANUALLY ADDING A SLASH):

 % ffmpeg -i
 "https://key.vscdns.com/?key_id=2&model_id=991011&&access_key="
 [https @ 00000262520c9b00] HTTP error 403 Forbidden
 }}}

 It affects any URL where the query string is immediate. Which means that
 it affects direct media links, M3U8/HLS playlists (where the user has no
 control over the URLs and no way to fix it themselves), etc.

 This is problematic because the ".com?" format is entirely up to the
 streaming media provider, and cannot be affected by the user. Any
 streaming playlist that uses a field such as "EXT-X-KEY" with such a URL
 (as in my example URL) will fail completely in FFmpeg and there's nothing
 the user can do about it. So it is imperative that FFmpeg understands
 RFC2396 and allows the slash to be optional.

 I looked at the parsing code and it seems quite simple to fix. It
 currently looks for the first "/" slash and "?" question mark. The fix
 would probably be to say "If a question mark was found, and it is earlier
 than the first slash, or no slash was found at all" then set the pointers
 so that it treats the "?" as the start of the query and the character
 ''immediately before that'' as the end of the domain name.

 PS: I have set the same priority as the earlier URL parsing error ticket I
 referred to above. Feel free to lower the priority.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8466>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker


More information about the FFmpeg-trac mailing list