#9020(avformat:new): http BUFFER_SIZE too small
#9020: http BUFFER_SIZE too small ----------------------------------+--------------------------------------- Reporter: jtojnar | Type: defect Status: new | Priority: normal Component: avformat | Version: unspecified Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ----------------------------------+--------------------------------------- Summary of the bug: When making a HTTP request with a large authorization cookie, the execution fails with “overlong headers” error. The limit was increased to 8192 bytes in https://github.com/FFmpeg/FFmpeg/commit/d29c42974487d5fa0a5c1b05a09da5c5818a... (part of ffmpeg 4.3.1) but that is still not enough for authorization headers required for downloading videos from Microsoft Stream (the cookie header alone is almost 8kiB). How to reproduce: {{{ $ ffmpeg -headers "Cookie: Authorization_Api=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%XX%XX; Signature_Api=XX%XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%XXXXXXXX%XXXX; UserSession_Api=XXXXXXXXX=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&XXXXXXX=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX; Authorization=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%XX%XX; Signature=XX%XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%XXXXXXXX%XXXX; Extra- dummy-data-representing-the-rest-of-headers: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX; XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -i "http://example.com/" ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers built with gcc 9.3.0 (GCC) configuration: --disable-static --prefix=/nix/store /n77nwda6g2b9iaxxwk8mdzardxzgz7dl-ffmpeg-4.3.1 --arch=x86_64 --target_os=linux --enable-gpl --enable-version3 --enable-shared --enable- pic --enable-runtime-cpudetect --enable-hardcoded-tables --enable-pthreads --disable-w32threads --disable-os2threads --enable-network --enable- pixelutils --enable-ffmpeg --disable-ffplay --enable-ffprobe --enable- avcodec --enable-avdevice --enable-avfilter --enable-avformat --enable- avresample --enable-avutil --enable-postproc --enable-swresample --enable- swscale --disable-doc --enable-libass --enable-bzlib --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libmp3lame --enable- iconv --enable-libtheora --enable-libssh --enable-vaapi --enable-libdrm --enable-vdpau --enable-libvorbis --enable-libvpx --enable-lzma --disable- opengl --disable-libmfx --disable-libaom --enable-libpulse --enable-sdl2 --enable-libsoxr --enable-libx264 --enable-libxvid --enable-zlib --enable- libopus --enable-libspeex --enable-libx265 --enable-libdav1d --disable- debug --enable-optimizations --disable-extra-warnings --disable-stripping libavutil 56. 51.100 / 56. 51.100 libavcodec 58. 91.100 / 58. 91.100 libavformat 58. 45.100 / 58. 45.100 libavdevice 58. 10.100 / 58. 10.100 libavfilter 7. 85.100 / 7. 85.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 7.100 / 5. 7.100 libswresample 3. 7.100 / 3. 7.100 libpostproc 55. 7.100 / 55. 7.100 [http @ 0x1f7e480] No trailing CRLF found in HTTP header. Adding it. [http @ 0x1f7e480] overlong headers http://example.com/: Invalid argument }}} I managed to fix it locally by patching ffmpeg with a bigger buffer size (https://github.com/ytdl-org/youtube-dl/pull/24649#issuecomment-738002126) but ideally, ffmpeg would use dynamic allocation for the headers. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small -------------------------------------+------------------------------------ Reporter: jtojnar | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: unspecified | Resolution: Keywords: | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | -------------------------------------+------------------------------------ Comment (by sskras): I debbuged the ytdl and found out that in case of my Microsoft Stream video it builds 9598 bytes long http header. Here it's structure, broken down and simplified: {{{ Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip, deflate Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.11 Safari/537.36 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Cookie: UserSession_Api=signature=XrL<...40-chars...>&payload=eyJF<...678-chars...>; Signature_Api=aMdcveStFNasMAa2%252fyLnFJ1oc%252bQnlJ7rbR6C5kU%253d; Authorization_Api=eyJU<...3278-chars...>; Authorization=eyJU<...3278-chars...>; Signature=aMdcveStFNasMAa2%252fyLnFJ1oc%252bQnlJ7rbR6C5kU%253d Authorization: Bearer eyJ0<...1692-chars...> }}} I found an article about the max. header size support for http requests: https://www.geekersdigest.com/max-http-request-header-size-server- comparison/ According to it, only five popular implementations use 8k or lower limit for the http header. Another five implementations use 16k limit. The rests six implementation use 32k or more (up to 2M on IIS 4.x). I think it's reasonable to increase the `HTTP_HEADERS_SIZE` to 16k (as is seen in my example). -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:1> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: jtojnar | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Changes (by cehoyos): * keywords: => http * version: unspecified => git-master -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:2> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: jtojnar | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Changes (by sskras): * Attachment "0001-libavformat-http.h-Increase-HTTP_HEADERS_SIZE.patch" added. libavformat/http.h: Increase HTTP_HEADERS_SIZE. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: jtojnar | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by sskras): I made a patch: https://github.com/sskras/FFmpeg/commit/f24d93109400999c30ae3a47194592a9db19... Sadly I am unable to build the whole project to test this. And I am not on a developers list yet too. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:3> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: jtojnar | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by cehoyos): The comment is both too long and unclear (what means ''here'' in the first sentence?) - remove the comment, add as much of it as you want to the commit message and send your patch - made with `git format-patch` - to the FFmpeg development mailing list where it can be reviewed. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:4> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: jtojnar | Owner: Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by cehoyos): There might be an issue with the stack size in http.c - in the past there was a (definitely not strict) goal of trying not to use too big structs on the stack, I am not sure how relevant this still is (and how relevant it is for network code). -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:5> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: Jan Tojnar | Owner: (none) Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Changes (by Jan Tojnar): * Attachment "0001-avformat-http-Increase-HTTP_HEADERS_SIZE.patch" added. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: Jan Tojnar | Owner: (none) Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by Jan Tojnar): To move this forward, I tried to address the comments here. No idea about stack size limitations, I guess the people on the mailing list will know more. I can also submit your patch to the mailing list for you if you confirm that it is your own work as described by https://developercertificate.org/ -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:6> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: Jan Tojnar | Owner: (none) Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by Piotr G): Please re-consider this patch. With object-based (S3) storage and potentially very long pre-signed URLs it is possible the URL can be larger than 5 kB. If fact only the X-Amz-Security-Token query-part of an STS- generated pre-signed url can be 4096 bytes. At the moment ffmpeg/ffprobe will cut off the exces from the URL leading to S3 server rejecting the pre-signed URL with a HTTP 400. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:7> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: Jan Tojnar | Owner: (none) Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by Balling): How is that out problem? Fix S3. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:8> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: Jan Tojnar | Owner: (none) Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by eb3f73): Replying to [comment:8 Balling]:
How is that out problem? Fix S3.
I don't think that's practical. Further, I have noticed at least one other site that suffers from this issue. Is there a technical reason why the length limit should not be increased? It's not defined by the standard (as far as I know), and various HTTP servers allow for headers of lengths 4k, 8k, 16k, and (in one case) 48k. [https://stackoverflow.com/questions/686217/maximum-on-http-header-values citation] -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:9> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
#9020: http BUFFER_SIZE too small ------------------------------------+------------------------------------ Reporter: Jan Tojnar | Owner: (none) Type: defect | Status: new Priority: normal | Component: avformat Version: git-master | Resolution: Keywords: http | Blocked By: Blocking: | Reproduced by developer: 0 Analyzed by developer: 0 | ------------------------------------+------------------------------------ Comment (by Timo R.): The buffer is stored on the stack, so increasing that size is a bad idea without refactoring to use a heap-allocated buffer first. -- Ticket URL: <https://trac.ffmpeg.org/ticket/9020#comment:10> FFmpeg <https://ffmpeg.org> FFmpeg issue tracker
participants (1)
-
FFmpeg