[FFmpeg-trac] #5080(undetermined:new): Excessive HTTP GETs reading MP4 from web server
FFmpeg
trac at avcodec.org
Thu Dec 10 20:27:24 CET 2015
#5080: Excessive HTTP GETs reading MP4 from web server
--------------------------------------+----------------------------------
Reporter: nealzebub | Type: defect
Status: new | Priority: normal
Component: undetermined | Version: 2.8.3
Keywords: http | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
--------------------------------------+----------------------------------
OS: RHEL 3.14.33-26.47.amzn1.x86_64
Build: ffmpeg-2.8.1-64bit-static
Obtained from: http://johnvansickle.com/ffmpeg/
== Summary of the bug:==
When FFmpeg/FFprobe reads an MP4 file from a web server that allows
partial download, the application may use an excessive number of GETs.
This occurs with files in which audio/video frames are interleaved where
audio-video frames exceed a difference in file offset, presumably
exceeding some I/O buffer size. I have encountered many files that bring
out this behavior, so it's not a corner case.
Doubling probesize to 10000000 does not decrease the number of GETs. All
these GETs can cause a dramatic slowdown of encoding speed(4fps vs.
90fps), depending upon the response time of the service/server.
To develop my presumption of frame "position", I used FFprobe's to print
that information.
{{{
./ffprobe -hide_banner -v trace -show_entries
packet=pos,pts_time,codec_type
"http://transcodes.s3.amazonaws.com/content/iPhone4_upload_via_3G.MP4" -of
compact 2>&1 | grep 'Content-Range\|codec_type'
}}}
And an excerpt from the attached output of that command:
{{{
packet|codec_type=audio|pts_time=3.436553|pos=42024
packet|codec_type=video|pts_time=2.466667|pos=61374
[http @ 0x3f7a4a0] header='Content-Range: bytes 31559-684616/684617'
packet|codec_type=audio|pts_time=3.482993|pos=42182
packet|codec_type=audio|pts_time=3.529433|pos=42317
packet|codec_type=video|pts_time=2.533333|pos=66016
packet|codec_type=audio|pts_time=3.575873|pos=42473
packet|codec_type=video|pts_time=2.600000|pos=66407
packet|codec_type=audio|pts_time=3.622313|pos=42618
packet|codec_type=video|pts_time=2.666667|pos=67537
packet|codec_type=audio|pts_time=3.668753|pos=48642
packet|codec_type=audio|pts_time=3.715193|pos=48785
packet|codec_type=video|pts_time=2.733333|pos=68161
[http @ 0x3f7a4a0] header='Content-Range: bytes 48926-684616/684617'
packet|codec_type=audio|pts_time=3.761633|pos=48926
packet|codec_type=video|pts_time=2.800000|pos=69005
packet|codec_type=audio|pts_time=3.808073|pos=49058
}}}
== How to reproduce: ==
''Please note that this is extremely dependent on the source file and must
be over HTTP, which is why I made it available.''
{{{
./ffmpeg -hide_banner -v trace -report -i
"https://transcodes.s3.amazonaws.com/content/iPhone4_upload_via_3G.MP4" -y
output.mp4 2>&1 | grep 'Content-Range\|frame='
}}}
== Log Output: ==
I had to use a fairly verbose logging level to show the http requests and
frames processed. Please see the attached report to this bug.
Here's an excerpt, though:
{{{
[https @ 0x48f2640] request: GET /content/iPhone4_upload_via_3G.MP4
HTTP/1.1
User-Agent: Lavf/56.40.101
Accept: */*
Range: bytes=65677-
Connection: close
Host: transcodes.s3.amazonaws.com
Icy-MetaData: 1
[libx264 @ 0x48fca20] frame= 0 QP=23.32 NAL=3 Slice:I Poc:0 I:112 P:0
SKIP:0 size=6416 bytes
[libx264 @ 0x48fca20] frame= 1 QP=23.35 NAL=2 Slice:P Poc:4 I:0
P:105 SKIP:7 size=1053 bytes
[libx264 @ 0x48fca20] frame= 2 QP=31.00 NAL=0 Slice:B Poc:2 I:0
P:65 SKIP:47 size=87 bytes
[libx264 @ 0x48fca20] frame= 3 QP=23.65 NAL=2 Slice:P Poc:8 I:0
P:99 SKIP:13 size=957 bytes
[libx264 @ 0x48fca20] frame= 4 QP=31.00 NAL=0 Slice:B Poc:6 I:0
P:56 SKIP:55 size=79 bytes
[libx264 @ 0x48fca20] frame= 5 QP=23.66 NAL=2 Slice:P Poc:12 I:0
P:104 SKIP:8 size=1133 bytes
[libx264 @ 0x48fca20] frame= 6 QP=29.67 NAL=0 Slice:B Poc:10 I:0
P:60 SKIP:47 size=134 bytes
[libx264 @ 0x48fca20] frame= 7 QP=23.84 NAL=2 Slice:P Poc:16 I:0
P:98 SKIP:14 size=884 bytes
[libx264 @ 0x48fca20] frame= 8 QP=31.00 NAL=0 Slice:B Poc:14 I:0
P:62 SKIP:49 size=84 bytes
[libx264 @ 0x48fca20] frame= 9 QP=23.73 NAL=2 Slice:P Poc:20 I:0
P:108 SKIP:4 size=1056 bytes
[https @ 0x48f2640] request: GET /content/iPhone4_upload_via_3G.MP4
HTTP/1.1
User-Agent: Lavf/56.40.101
Accept: */*
Range: bytes=82048-
Connection: close
Host: transcodes.s3.amazonaws.com
Icy-MetaData: 1
[libx264 @ 0x48fca20] frame= 10 QP=31.00 NAL=0 Slice:B Poc:18 I:0
P:48 SKIP:62 size=68 bytes
[libx264 @ 0x48fca20] frame= 11 QP=23.79 NAL=2 Slice:P Poc:24 I:0
P:107 SKIP:5 size=1079 bytes
[libx264 @ 0x48fca20] frame= 12 QP=31.50 NAL=0 Slice:B Poc:22 I:0
P:48 SKIP:64 size=56 bytes
[libx264 @ 0x48fca20] frame= 13 QP=23.96 NAL=2 Slice:P Poc:28 I:0
P:105 SKIP:7 size=971 bytes
[libx264 @ 0x48fca20] frame= 14 QP=30.33 NAL=0 Slice:B Poc:26 I:0
P:69 SKIP:39 size=137 bytes
[libx264 @ 0x48fca20] frame= 15 QP=23.62 NAL=2 Slice:P Poc:32 I:0
P:112 SKIP:0 size=1858 bytes
[libx264 @ 0x48fca20] frame= 16 QP=29.60 NAL=0 Slice:B Poc:30 I:0
P:64 SKIP:46 size=196 bytes
[libx264 @ 0x48fca20] frame= 17 QP=23.32 NAL=2 Slice:P Poc:36 I:0
P:105 SKIP:7 size=1228 bytes
[libx264 @ 0x48fca20] frame= 18 QP=29.05 NAL=0 Slice:B Poc:34 I:0
P:83 SKIP:27 size=170 bytes
[libx264 @ 0x48fca20] frame= 19 QP=23.24 NAL=2 Slice:P Poc:40 I:0
P:107 SKIP:5 size=1039 bytes
[libx264 @ 0x48fca20] frame= 20 QP=29.67 NAL=0 Slice:B Poc:38 I:0
P:74 SKIP:33 size=181 bytes
[libx264 @ 0x48fca20] frame= 21 QP=23.45 NAL=2 Slice:P Poc:44 I:0
P:110 SKIP:2 size=1133 bytes
[https @ 0x48f2640] request: GET /content/iPhone4_upload_via_3G.MP4
HTTP/1.1
User-Agent: Lavf/56.40.101
Accept: */*
Range: bytes=90655-
Connection: close
Host: transcodes.s3.amazonaws.com
Icy-MetaData: 1
}}}
--
Ticket URL: <https://trac.ffmpeg.org/ticket/5080>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list