Syncing clips based on audio waveform?
Hi all, This is coming from out of left field I imagine, but I'm wondering if anyone knows of a filter (or chain of filters) which would allow me to align/sync two video clips based on matches in their audio waveforms. I'm shooting events with multiple cameras and would like to either trim the clips to a precise, matching starting frame or generate a metadata "offset" marker indicating how far off one clip is from the other in time. Then I can trim the clips to match. Is this possible in ffmpeg (or anywhere else for that matter!)? Thanks very much! Art
On 3/1/2013 9:38 AM, Arthur Wait wrote:
Hi all,
This is coming from out of left field I imagine, but I'm wondering if anyone knows of a filter (or chain of filters) which would allow me to align/sync two video clips based on matches in their audio waveforms.
I'm shooting events with multiple cameras and would like to either trim the clips to a precise, matching starting frame or generate a metadata "offset" marker indicating how far off one clip is from the other in time. Then I can trim the clips to match.
Is this possible in ffmpeg (or anywhere else for that matter!)?
I don't know if there is anything in ffmpeg that would accomplish what you need right now, and I doubt implementing it as a filter is very practical. It could be implemented as a separate utility -- give it a reference file and another file and it spits out time offset of the second file relative to the reference file (if it can find a correlation). Pavel.
On 3/1/2013 3:17 PM, Pavel Koshevoy wrote:
On 3/1/2013 9:38 AM, Arthur Wait wrote:
Hi all,
This is coming from out of left field I imagine, but I'm wondering if anyone knows of a filter (or chain of filters) which would allow me to align/sync two video clips based on matches in their audio waveforms.
I'm shooting events with multiple cameras and would like to either trim the clips to a precise, matching starting frame or generate a metadata "offset" marker indicating how far off one clip is from the other in time. Then I can trim the clips to match.
Is this possible in ffmpeg (or anywhere else for that matter!)?
I don't know if there is anything in ffmpeg that would accomplish what you need right now, and I doubt implementing it as a filter is very practical. It could be implemented as a separate utility -- give it a reference file and another file and it spits out time offset of the second file relative to the reference file (if it can find a correlation).
Now that I think about it, this feature probably should be implemented in ffprobe. Pavel
On Mar 1, 2013, at 6:00 PM, Pavel Koshevoy <pkoshevoy@gmail.com> wrote:
On 3/1/2013 3:17 PM, Pavel Koshevoy wrote:
On 3/1/2013 9:38 AM, Arthur Wait wrote:
Hi all,
This is coming from out of left field I imagine, but I'm wondering if anyone knows of a filter (or chain of filters) which would allow me to align/sync two video clips based on matches in their audio waveforms.
I'm shooting events with multiple cameras and would like to either trim the clips to a precise, matching starting frame or generate a metadata "offset" marker indicating how far off one clip is from the other in time. Then I can trim the clips to match.
Is this possible in ffmpeg (or anywhere else for that matter!)?
I don't know if there is anything in ffmpeg that would accomplish what you need right now, and I doubt implementing it as a filter is very practical. It could be implemented as a separate utility -- give it a reference file and another file and it spits out time offset of the second file relative to the reference file (if it can find a correlation).
Now that I think about it, this feature probably should be implemented in ffprobe.
+1 for that. Dave Rice
On 3/2/13, Dave Rice <dave@dericed.com> wrote:
On Mar 1, 2013, at 6:00 PM, Pavel Koshevoy <pkoshevoy@gmail.com> wrote:
On 3/1/2013 3:17 PM, Pavel Koshevoy wrote:
On 3/1/2013 9:38 AM, Arthur Wait wrote:
Hi all,
This is coming from out of left field I imagine, but I'm wondering if anyone knows of a filter (or chain of filters) which would allow me to align/sync two video clips based on matches in their audio waveforms.
I'm shooting events with multiple cameras and would like to either trim the clips to a precise, matching starting frame or generate a metadata "offset" marker indicating how far off one clip is from the other in time. Then I can trim the clips to match.
Is this possible in ffmpeg (or anywhere else for that matter!)?
I don't know if there is anything in ffmpeg that would accomplish what you need right now, and I doubt implementing it as a filter is very practical. It could be implemented as a separate utility -- give it a reference file and another file and it spits out time offset of the second file relative to the reference file (if it can find a correlation).
Now that I think about it, this feature probably should be implemented in ffprobe.
+1 for that.
Nonsense...
Dave Rice _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-user
----- Original Message ----- From: "Arthur Wait" <artwait@gmail.com> To: <ffmpeg-user@ffmpeg.org> Sent: Friday, March 01, 2013 5:38 PM Subject: [FFmpeg-user] Syncing clips based on audio waveform?
Hi all,
This is coming from out of left field I imagine, but I'm wondering if anyone knows of a filter (or chain of filters) which would allow me to align/sync two video clips based on matches in their audio waveforms.
I'm guessing you're after field recording syncing... it is done in a commercial application called Plural Eyes. The original developer has sold it, and now it's owned (and improved / maintained) by Red Giant IIRC. With improved, i mean it sucks less, but it still is dog slow, and has only success when both sources are very much alike. For double system shooting it is absolutely unreliable (It is now advertised as a 'Fix it in post' product.)
From my (limited) point of view, waveform is not the best way, as noise will drive you crazy. Finding peaks would be better, but you'll find peaks in one source that is not in the other. (Like tapping on mics, other close sounds.) Then, there is no guarantee both sources run at exactly the same speed. A drift of some 20 Msecs over a long take is not uncommon. Not really a problem for post, but you need to compensate in your filter. Then there is the trouble of not knowing for sure what takes belong together. (Of course no problem in a lab, but for stuff coming out of the field, Murphy has a strong case.)
hth, Bouke (who is a timecode freak)
I'm shooting events with multiple cameras and would like to either trim the clips to a precise, matching starting frame or generate a metadata "offset" marker indicating how far off one clip is from the other in time. Then I can trim the clips to match.
Is this possible in ffmpeg (or anywhere else for that matter!)?
Thanks very much!
Art _______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-user
yes ! you can do it with a new filter. https://www.emirchouchane.com/async-v0.1-ffmpeg/ -- Sent from: http://www.ffmpeg-archive.org/
2017-10-05 21:10 GMT+02:00 Emir <emir.chouchane@gmail.com>:
yes ! you can do it with a new filter. https://www.emirchouchane.com/async-v0.1-ffmpeg/
Is there a reason you don't send a patch (made with git format-patch) with your new filter to the FFmpeg development mailing list? Thank you, Carl Eugen
participants (7)
-
Arthur Wait -
Bouke (VideoToolShed) -
Carl Eugen Hoyos -
Dave Rice -
Emir -
Paul B Mahol -
Pavel Koshevoy