iMovie created variable frame rates cause a/v sync problems
I have a file generated from Apple's "iMovie HD". It has a variable frame rate, according to mediainfo: Frame rate mode : Variable Frame rate : 29.402 fps Minimum frame rate : 0.303 fps Maximum frame rate : 29.970 fps Original frame rate : 29.970 fps I am trying to get ffmpeg to create a transcoded version suitable for web & iphone streaming, but the audio & video are out of sync. I used ffmpeg -i original.m4v -acodec copy -vcodec libx264 output.mp4 I also tried with -acodec libfaac. Same result. Oddly enough, mediainfo on the ffmpeg converted file shows the same frame rate information, but the soundtrack is clearly not in sync: Frame rate mode : Variable Frame rate : 29.402 fps Minimum frame rate : 0.303 fps Maximum frame rate : 29.970 fps Original frame rate : 29.970 fps Is there some simple command line option that I'm missing to keep the two in sync? I can provide the original file if needed. The current version of iMovie does not have the same problem because it outputs at constant frame rate by default. -jsd-
Hi, On 3/16/2009 2:36 PM, Jon Drukman wrote:
I have a file generated from Apple's "iMovie HD". It has a variable frame rate, according to mediainfo:
Frame rate mode : Variable Frame rate : 29.402 fps Minimum frame rate : 0.303 fps Maximum frame rate : 29.970 fps Original frame rate : 29.970 fps
I am trying to get ffmpeg to create a transcoded version suitable for web & iphone streaming, but the audio & video are out of sync.
I used
ffmpeg -i original.m4v -acodec copy -vcodec libx264 output.mp4
I also tried with -acodec libfaac. Same result.
Oddly enough, mediainfo on the ffmpeg converted file shows the same frame rate information, but the soundtrack is clearly not in sync:
Frame rate mode : Variable Frame rate : 29.402 fps Minimum frame rate : 0.303 fps Maximum frame rate : 29.970 fps Original frame rate : 29.970 fps
Is there some simple command line option that I'm missing to keep the two in sync? I can provide the original file if needed.
The current version of iMovie does not have the same problem because it outputs at constant frame rate by default.
First, please post full ffmpeg commandline, and ffmpeg output. Use latest svn, also run it using -v 3 option. Then upload your sample according to http://www.ffmpeg.org/bugreports.html Thanks. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA checking for life_signs in -lkenny... no FFmpeg maintainer http://www.ffmpeg.org
Baptiste Coudurier wrote:
ffmpeg -i original.m4v -acodec copy -vcodec libx264 output.mp4
I also tried with -acodec libfaac. Same result.
First, please post full ffmpeg commandline, and ffmpeg output. Use latest svn, also run it using -v 3 option.
command line from my original message is above. i built a brand new libx264 and ffmpeg today from the latest snapshots/svn sources. output is: FFmpeg version SVN-r18017, Copyright (c) 2000-2009 Fabrice Bellard, et al. configuration: --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libx264 libavutil 50. 0. 0 / 50. 0. 0 libavcodec 52.21. 0 / 52.21. 0 libavformat 52.31. 1 / 52.31. 1 libavdevice 52. 1. 0 / 52. 1. 0 libswscale 0. 7. 1 / 0. 7. 1 built on Mar 16 2009 11:42:05, gcc: 4.0.1 (Apple Inc. build 5484) Seems stream 1 codec frame rate differs from container frame rate: 59.94 (5994/100) -> 29.97 (30000/1001) Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Movies/iMovie Events.localized/Misc Junk/21703_10_1237137901.m4v': Duration: 00:02:55.75, start: 0.000000, bitrate: 2619 kb/s Stream #0.0(eng): Audio: aac, 44100 Hz, stereo, s16 Stream #0.1(eng): Video: h264, yuv420p, 720x480, 29.97 tbr, 29.97 tbn, 59.94 tbc [libx264 @ 0x1022e00]using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64 [libx264 @ 0x1022e00]profile Baseline, level 3.0 Output #0, mp4, to 'output.mp4': Stream #0.0(eng): Video: libx264, yuv420p, 720x480, q=2-31, 200 kb/s, 30k tbn, 29.97 tbc Stream #0.1(eng): Audio: libfaac, 44100 Hz, stereo, s16 Stream mapping: Stream #0.1 -> #0.0 Stream #0.0 -> #0.1 Press [q] to stop encoding frame= 5070 fps= 29 q=-1.0 Lsize= 37928kB time=172.44 bitrate=1801.8kbits/s video:35129kB audio:2698kB global headers:0kB muxing overhead 0.267242% [libx264 @ 0x1022e00]slice I:423 Avg QP:26.92 size: 18265 [libx264 @ 0x1022e00]slice P:4647 Avg QP:29.33 size: 6078 [libx264 @ 0x1022e00]mb I I16..4: 29.6% 0.0% 70.4% [libx264 @ 0x1022e00]mb P I16..4: 13.3% 0.0% 0.0% P16..4: 59.3% 0.0% 0.0% 0.0% 0.0% skip:27.4% [libx264 @ 0x1022e00]final ratefactor: 43.48 [libx264 @ 0x1022e00]SSIM Mean Y:0.9429902 [libx264 @ 0x1022e00]kb/s:1701.1
Then upload your sample according to http://www.ffmpeg.org/bugreports.html
i put the sample on http://s3.thismoment.com/ffmpeg_sample.m4v - i hope that's ok. it's 57MB and the problem isn't really obvious until the end of the video when you can see the people's mouths (although the sound is out of sync the whole way through). this was the smallest example file i could find at the moment. -jsd-
Calling all FFMPEG developers/experts. My company is prepared to offer a bounty of up to US$500 to the person who can provide a patch and working command line to solve the problem described in the thread quoted below. To claim the full bounty you must provide: 1. A patch that is accepted into the main ffmpeg source repository. 2. A command line that can generate a file with proper a/v sync maintained throughout that will play in a web based flash player and on the Apple iPhone. Partial bounty will be awarded if: 1. The patch is not acceptable to the ffmpeg maintainers but compiles on Mac OS X and Red Hat Enterprise Linux 5 at minimum, and solves my problem. (minus $100) 2. A third-party tool like MP4Box is still required to mux up a working file. (minus $50) You can use the file listed in the original thread below, and here is a second, larger file, which exhibits the problem at the beginning (maybe easier to work with since you only need to encode the first minute). http://s3.thismoment.com/bolivia.m4v Contact me directly via email if you want to discuss the details. Jon Drukman wrote:
Baptiste Coudurier wrote:
ffmpeg -i original.m4v -acodec copy -vcodec libx264 output.mp4
I also tried with -acodec libfaac. Same result.
First, please post full ffmpeg commandline, and ffmpeg output. Use latest svn, also run it using -v 3 option.
command line from my original message is above. i built a brand new libx264 and ffmpeg today from the latest snapshots/svn sources.
output is:
FFmpeg version SVN-r18017, Copyright (c) 2000-2009 Fabrice Bellard, et al. configuration: --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libx264 libavutil 50. 0. 0 / 50. 0. 0 libavcodec 52.21. 0 / 52.21. 0 libavformat 52.31. 1 / 52.31. 1 libavdevice 52. 1. 0 / 52. 1. 0 libswscale 0. 7. 1 / 0. 7. 1 built on Mar 16 2009 11:42:05, gcc: 4.0.1 (Apple Inc. build 5484)
Seems stream 1 codec frame rate differs from container frame rate: 59.94 (5994/100) -> 29.97 (30000/1001) Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Movies/iMovie Events.localized/Misc Junk/21703_10_1237137901.m4v': Duration: 00:02:55.75, start: 0.000000, bitrate: 2619 kb/s Stream #0.0(eng): Audio: aac, 44100 Hz, stereo, s16 Stream #0.1(eng): Video: h264, yuv420p, 720x480, 29.97 tbr, 29.97 tbn, 59.94 tbc [libx264 @ 0x1022e00]using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64 [libx264 @ 0x1022e00]profile Baseline, level 3.0 Output #0, mp4, to 'output.mp4': Stream #0.0(eng): Video: libx264, yuv420p, 720x480, q=2-31, 200 kb/s, 30k tbn, 29.97 tbc Stream #0.1(eng): Audio: libfaac, 44100 Hz, stereo, s16 Stream mapping: Stream #0.1 -> #0.0 Stream #0.0 -> #0.1 Press [q] to stop encoding frame= 5070 fps= 29 q=-1.0 Lsize= 37928kB time=172.44 bitrate=1801.8kbits/s video:35129kB audio:2698kB global headers:0kB muxing overhead 0.267242% [libx264 @ 0x1022e00]slice I:423 Avg QP:26.92 size: 18265 [libx264 @ 0x1022e00]slice P:4647 Avg QP:29.33 size: 6078 [libx264 @ 0x1022e00]mb I I16..4: 29.6% 0.0% 70.4% [libx264 @ 0x1022e00]mb P I16..4: 13.3% 0.0% 0.0% P16..4: 59.3% 0.0% 0.0% 0.0% 0.0% skip:27.4% [libx264 @ 0x1022e00]final ratefactor: 43.48 [libx264 @ 0x1022e00]SSIM Mean Y:0.9429902 [libx264 @ 0x1022e00]kb/s:1701.1
Then upload your sample according to http://www.ffmpeg.org/bugreports.html
i put the sample on http://s3.thismoment.com/ffmpeg_sample.m4v - i hope that's ok. it's 57MB and the problem isn't really obvious until the end of the video when you can see the people's mouths (although the sound is out of sync the whole way through). this was the smallest example file i could find at the moment.
-jsd-
On an unrelated note:
ffmpeg -i original.m4v -acodec copy -vcodec libx264 output.mp4
You should *always* be using presets with x264, at least as a starting point (e.g. -vpre hq). Dark Shikari
On 20/3/09 02:40, Jason Garrett-Glaser wrote:
On an unrelated note:
ffmpeg -i original.m4v -acodec copy -vcodec libx264 output.mp4
You should *always* be using presets with x264, at least as a starting point (e.g. -vpre hq).
Note that -vpre has to come after the -vcodec option. ffmpeg -i original.m4v -acodec copy -vcodec libx264 -vpre hq -b 1M output.mp4 If you want some advice regarding x264 encoding in FFmpeg, you can take a look at my guide on: http://rob.opendot.cl/index.php/useful-stuff/ffmpeg-x264-encoding-guide/ The presets are included in the 0.5 release and in current svn. They are installed to ${prefix}/share/ffmpeg so by default to /usr/local/share/ffmpeg. One can also pass absolute paths to -vpre, e.g. -vpre /usr/local/share/ffmpeg/libx264-hq.ffpreset I don't really know about a/v sync as that's a time stamp/container issue, not really a codec issue. I hope that helps with part of your problems. Regards, Rob
Robert Swain wrote:
If you want some advice regarding x264 encoding in FFmpeg, you can take a look at my guide on:
http://rob.opendot.cl/index.php/useful-stuff/ffmpeg-x264-encoding-guide/
thanks, i've actually already figured out the x264 side of things. that part all works fine. you can see the results live here http://www.thismoment.com/moment/view/21658/bolivia works great in browser, also viewable on iphone. but notice the huge a/v sync problem... only problematic when the source file has a variable frame rate and is generated with specific quicktime encoding options. this one (also converted from iMovie but with different settings) has no sync problems: http://www.thismoment.com/moment/view/21588/queen-of-the-jungle the a/v sync issue is the deal breaker, and i wanted to point out that it happens no matter what you do with the x264 encoding options. (in fact you probably don't even have to use libx264 to see the broken behavior.) -jsd-
Jon Drukman wrote:
the a/v sync issue is the deal breaker, and i wanted to point out that it happens no matter what you do with the x264 encoding options. (in fact you probably don't even have to use libx264 to see the broken behavior.)
Have you looked at adding in a "-async 2" command - before the -vcodec portion of your command line? In many cases, this will help tremendously. Ron Gage
-jsd-
_______________________________________________ ffmpeg-user mailing list ffmpeg-user at mplayerhq.hu https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
Ron Gage wrote:
Jon Drukman wrote:
the a/v sync issue is the deal breaker, and i wanted to point out that it happens no matter what you do with the x264 encoding options. (in fact you probably don't even have to use libx264 to see the broken behavior.)
Have you looked at adding in a "-async 2" command - before the -vcodec portion of your command line? In many cases, this will help tremendously.
-async 2 doesn't have any effect on it at all. thanks! keep 'em coming... -jsd-
I come across video of variable frame rate when I tried to encode video with the constant frame rate of 15 FPS into video with the constant frame rate of 24 FPS. I typically create batch files to let command interpreter execute a series of command sequence, instead of entering only a line of command once at a time. ffmpeg -i "input.mkv" -r "24" -vcodec "libx264" -crf "0" "output.mkv" Unknown to me, however, ffmpeg would encode the video into a video of variable frame rate when the frame rate of the source is lower than output video. The unintended consequence would be that the output video is out of sync with audio. Without changing the “playback speed”, changing frame rate itself will inevitably result in the change of video length, even though such difference may be negligible. “Out of sync” is obviously caused by unintended change of playback speed of the output video. Of course, a video of variable frame rate can be created by joining videos of different (constant) frame rate together. The solution is obviously to avoid unintended change of playback speed of the output video. If I can make the output video having constant frame rate, I thought that the output video might be able to be in sync with audio. If the input video is of constant frame rate, it is possible to convert it to a series of image sequence. mkdir "output" ffmpeg -i "input.mkv" "output\f_%%1d.png" What if the input video is of variable frame rate? It turns out the image sequence cannot have variable frame rate. I specifically joining three videos together; each of them with the length of one second; first is of 15 FPS; second is of 24 FPS; third is of 30 FPS. Media Player Classic reports the joined video is of 15 FPS and variable frame rate. The average frame rate of the joined video would be 23 FPS; but Media Player Classic only reports 15 FPS, which is the frame rate for the first second. When I use the commands: mkdir "output" ffmpeg -i "input.mkv" "output\f_%%1d.png" ffmpeg actually makes the image sequence of 15 FPS when no frame rate as output option is being specified. In this case, it actually needs to drop frames to achieve the frame rate of 15 FPS. When it comes to suitable frame rate, it’s worth mentioning that the basic notion of video is that human eyes can only discriminate up to 12-15 frames every second, subject to light condition and other factors; the film industry commonly adopts 24 FPS as the standard frame rate. The short answer would be: mkdir "output" ffmpeg -i "input.mkv" -r "24" "output\f_%%1d.png" ffmpeg -r "24" -i "output\f_%%1d.png" -vcodec "libx264" -crf "0" "output.mkv" rmdir /s /q "output" Here I use Constant Rate Factor of “0”, which is lossless encoding of x264. -- View this message in context: http://ffmpeg-users.933282.n4.nabble.com/iMovie-created-variable-frame-rates... Sent from the FFmpeg-users mailing list archive at Nabble.com.
participants (7)
-
baptiste.coudurier＠gmail.com -
darkshikari＠gmail.com -
jdrukman＠gmail.com -
jsd＠cluttered.com -
robert.swain＠gmail.com -
ron＠rongage.org -
thljcl