[FFmpeg-user] Using segment and concat with multiple videos containing LTC audio on CH1
robertlazarski
robertlazarski at gmail.com
Fri May 4 17:31:14 EEST 2018
Hello all, first post.
I am trying to segment mov videos from multiple Zoom Q8 cameras with
exactly the same specs, by using the Linux command ltcdump, bash and
ffmpeg.
The LTC is being generated by a Tentacle Sync for each camera, that was
"jammed" to a Zoom F8 which is master audio I use to sync the final video
via LTC ... I have this working fine on each camera individually using
Ardour.
The goal is to concat the segments from these cameras into one video. The
main idea is the videos switch cameras at a fixed duration determined at
runtime by a simple constant like 20 seconds, determined by reading the
output from ltcdump on each video.
I have this very close to working. The last step is fixing an audio drift
problem caused during the transition from one camera to another. This drift
is because I cannot figure out how to exactly match an LTC audio frame to
each segment. I tried lots of numbers for the segment and was unsuccessful.
Here is an example of a transition and the problem I am trying to fix. Q81
is camera 1, Q82 is camera 2. I am missing frames 28 and 29 - the last .xx
digits are the frame and not milliseconds. And the LTC frame of somewhere
between 1597 and 1601 samples at 48KHZ does not match the segment exactly.
### ltcdump output from file q81_out280.mov.wav , last two .xx digits are
frames
#User bits Timecode | Pos. (samples)
#DISCONTINUITY
00000000 00:26:34:27 | 1221 2821
### ltcdump output from file q82_out305.mov.wav , last two .xx digits are
frames
#User bits Timecode | Pos. (samples)
#DISCONTINUITY
00000000 00:26:35:00 | 818 2418
What I believe I need to do is get the segment to match the LTC frame as
closely as possible. In these examples the segment I believe I need is 1600
/ 48000. However I tried and could not get the segments to match.
Here's my code to show what I am doing. Its about 300 lines.
#!/bin/bash
# This script generates a single video from parts of videos from 2 cameras
by using ltc.
# Videos are mov at 1280x740 with audio at 48KHZ and 16 bits
#
# usage:
# /home/myuser/input> ls
# output parseLTC.sh q81.mov q82.mov
# /home/myuser/input> cd output/
# /home/myuser/input/output> sh ../parseLTC.sh
# clear any files from previous processing
rm -f *.txt
rm -f *all*
rm -f *out*
# Trim first camera so the LTC frame starts as closely as possible to zero
# 00000000 00:26:15:00 | 2 1615
ffmpeg -y -v error -ss 00:00:06.067697 -t 00:30:15.365 -i ../q81.mov -c
copy q81t.mov
mv q81t.mov q81.mov
mv ../q82.mov q82.mov
# debug
ffmpeg -y -v error -i q81.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 q81.wav
ffmpeg -y -v error -i q82.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 q82.wav
# split file by key frames in a .07 second duration, need a couple entries
that
# ltcdump can find in each file - ideally just a single entry.
# first Q8
ffmpeg -y -v error -fflags +genpts -i q81.mov -vsync 1 -c:a pcm_s16le -c:v
libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720
-b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2
-ar 48000 -af "aresample=async=1:min_hard_comp=0.100000:first_pts=0"
-segment_time 0.07 -segment_list q81t.ffcat -sc_threshold 0
-force_key_frames "expr:gte(t,n_forced*0.07)" -f segment -reset_timestamps
1 -crf 0 q81_out%03d.mov
# second Q8
ffmpeg -y -v error -fflags +genpts -i ../q82.mov -vsync 1 -c:a pcm_s16le
-c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s
1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt
s16 -ac 2 -ar 48000 -af
"aresample=async=1:min_hard_comp=0.100000:first_pts=0" -segment_time 0.07
-segment_list q82t.ffcat -sc_threshold 0 -force_key_frames
"expr:gte(t,n_forced*0.07)" -f segment -reset_timestamps 1 -crf 0
q82_out%03d.mov
# save the ffcat output to run again if need be while debugging
cp q81t.ffcat sav_q81t.ffcat
cp q82t.ffcat sav_q82t.ffcat
# some errors I found while playing option bingo with ffmpeg
validate() {
fileToProcess=''
fileToProcess=$1
epoch_secs_ltc_to_process=0
ltc_to_process_formatted=''
isLTCDumpOutputValid=0
found_stop_time=0
# test for shell error
printf "\nExecuting: ltcdump -f 29.97 "$fileToProcess" "
# adding frame rate removes this DISCONTINUITY
# 00000000 00:26:53:24 | 7224 8824
#DISCONTINUITY
ltcdump -f 29.97 "${fileToProcess}" > /dev/null 2>&1
if [ $? -ne 0 ] ; then
printf "\nError processing file %s\n" "${fileToProcess}"
printf "\nshell returned non zero value on validity check"
return
fi
printf "\nExecuting: ltcdump -f 29.97 "${fileToProcess}" 2>/dev/null |
grep -v '#' | grep ':' | wc -w "
hasLTC=0
hasLTC=`ltcdump -f 29.97 "${fileToProcess}" 2>/dev/null | grep -v '#' |
grep ':' | wc -w`
if [[ $hasLTC -eq 0 ]]; then
printf "\nError processing file %s\n" "${fileToProcess}"
printf "\ncannot parse timecode\n"
return
fi
rm -f ltc.txt
# find timecode in format 'HH:MM:SS.00' with frames as last 2 digits
used for .00 time.
# remove occasional '.' in the form 00:00:00.0 instead of 00:00:00:0
# remove negative values
# disallow 0000000f 00:12:25:45
printf "\nExecuting: ltcdump -fps 29.97 "${fileToProcess}" 2>/dev/null
| grep -v '#' | grep 00000000 | grep -v '\.' | grep -v '-' > ltc.txt"
ltcdump -f 29.97 "${fileToProcess}" 2>/dev/null | grep -v '#' | grep
00000000 | grep -v '\.' | grep -v '-' > ltc.txt
lastTimecodeInSegment=0
lastTimecodeInSegment=`tail -n 1 ltc.txt | awk '{ print $2 }' | sed
's/\(.*\):/\1./' `
# can sometimes receive unparsable dates
if [[ ${#lastTimecodeInSegment} -ne 11 ]]; then
lastTimecodeInSegment=0
printf "\nError processing file %s\n" "${fileToProcess}"
printf "\ntimecode number of chars is not correct: "
printf " %s" "${#lastTimecodeInSegment}"
return
fi
# can receive invalid time such as: 00000680 00:13:61:10 |
22 1609
if ! date -d "$lastTimecodeInSegment" >/dev/null 2>&1; then
printf "\nError processing file %s\n" "${fileToProcess}"
printf "\ninvalid date format:" printf " %s"
"$lastTimecodeInSegment"
return
fi
epoch_secs_ltc_to_process=$(date +%s --date="$timecode_date
${lastTimecodeInSegment}")
ltc_to_process_formatted="$timecode_date ${lastTimecodeInSegment}"
printf "\nltc_to_process_formatted: $ltc_to_process_formatted"
if [[ $epoch_secs_ltc_to_process -ne 0 && $epoch_secs_stop_time -ne 0
&& $epoch_secs_ltc_to_process -ge $epoch_secs_stop_time ]]; then
found_stop_time=1
printf "\nskipping file %s\n" "${fileToProcess}"
printf "\ntimecode found:"
printf " %s" "${ltc_to_process_formatted}"
printf "\nis after or equal to stop time: "
printf " %s" "${epoch_secs_stop_time_formatted}"
return
fi
printf "\nfileToProcess seems valid:" "${fileToProcess}"
isLTCDumpOutputValid=1
}
# define timecode start time, just add date
timecode_date=2018-05-02
# skip segments before this timecode
epoch_secs_start_time=$(date +%s --date="$timecode_date 00:26:15")
start_time_formatted="$timecode_date 00:26:15.00"
# define timecode stop time
epoch_secs_stop_time=$(date +%s --date="$timecode_date 00:29:00")
stop_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000
$epoch_secs_stop_time seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' `
stop_time_formatted="$timecode_date ${stop_timecode}"
timeDivisionInSeconds=20
let epoch_secs_timecode_to_find=$((epoch_secs_start_time +
timeDivisionInSeconds))
find_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000
$epoch_secs_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./'
`
find_timecode_formatted="$timecode_date ${find_timecode}"
let
epoch_secs_next_timecode_to_find=$epoch_secs_timecode_to_find+$timeDivisionInSeconds
next_timecode_to_find=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000
$epoch_secs_next_timecode_to_find seconds" | cut -b1-11 | sed
's/\(.*\):/\1./' `
next_timecode_to_find_formatted="$timecode_date ${next_timecode_to_find}"
q81HasLTCStartTime=0
hasRejectBeforeTime=0
q81TimeCodeInProgress=0
q82TimeCodeInProgress=0
saveQ81CurrentTimeCodeInProgressAlreadyProcessed=0
rejectBeforeTime_formatted=''
saveQ81CurrentTimeCodeInProgress_to_find_formatted=''
while read i; do
# skip auto generated version line
if [[ $i =~ .*ffconcat.* ]]; then
continue
fi
q81VideoFile=`echo $i | cut -d " " -f2-`;
if [[ ! -f "${q81VideoFile}.wav" ]]; then
# convert q81 video segments to wav,
# so ltcdump can find the timecode
ffmpeg -v error -nostdin -i $q81VideoFile -vn -acodec pcm_s16le -ar
48000 -ac 2 "${q81VideoFile}.wav"
fi
validate "${q81VideoFile}.wav"
if [[ $found_stop_time -eq 1 ]]; then
printf "\nq81 video is after stop time, executing break\n"
printf "%s\n" "${q81VideoFile}"
break
fi
if [[ $q81HasLTCStartTime -eq 0 && $isLTCDumpOutputValid -eq 0 ]]; then
printf "\non q81HasLTCStartTime=0 and isLTCDumpOutputValid=0 .. q81
video is the default, occasional bad timecode values are ok"
printf "\nadding video to file list to send to all.mov: "
printf "file %s\n" "${q81VideoFile}"
printf "file %s\n" "${q81VideoFile}" >> all.txt
printf "%s\n" "${q81VideoFile}.wav" >> debug.txt
continue
fi
if [[ $q81HasLTCStartTime -eq 0 && $isLTCDumpOutputValid -eq 1 ]]; then
dateDiff=0
dateDiff=`date -d "$ltc_to_process_formatted $(date -d
"$start_time_formatted" +%s.%N) seconds ago" +%s.%N`
if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff"
then
q81HasLTCStartTime=1
printf "\non dateDiff $dateDiff , ltc_to_process_formatted
$ltc_to_process_formatted is later than or equal to start_time_formatted
$start_time_formatted"
printf "\nproceeding on file %s\n" "${q81VideoFile}"
else
printf "\non dateDiff $dateDiff , ltc_to_process_formatted
$ltc_to_process_formatted is before start_time_formatted
$start_time_formatted"
printf "\nskipping file %s\n" "${q81VideoFile}"
continue
fi
fi
if [[ $isLTCDumpOutputValid -eq 0 && $q81TimeCodeInProgress -eq 1 ]];
then
printf "\nq81 video is the default, occasional bad timecode values
are ok"
printf "\nadding video to file list to send to all.mov: "
printf "file %s\n" "${q81VideoFile}"
printf "file %s\n" "${q81VideoFile}" >> all.txt
printf "%s\n" "${q81VideoFile}.wav" >> debug.txt
continue
fi
if [[ $isLTCDumpOutputValid -eq 0 && $q81TimeCodeInProgress -eq 0 ]];
then
printf "\nskipping q81 video file , ltc is invalid and
q81TimeCodeInProgress is false: "
printf "file %s\n" "${q81VideoFile}"
continue
fi
if [[ $hasRejectBeforeTime -eq 1 ]]; then
dateDiff=0
dateDiff=`date -d "$ltc_to_process_formatted $(date -d
"$rejectBeforeTime_formatted" +%s.%N) seconds ago" +%s.%N`
if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0"
then
printf "\nexecuting continue on file "${q81VideoFile}" ,
dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is
before or equal to rejectBeforeTime_formatted $rejectBeforeTime_formatted
\n"
continue
else
printf "\nproceeding on file "${q81VideoFile}" , dateDiff
$dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is after
$rejectBeforeTime_formatted \n"
fi
fi
dateDiff=0
dateDiff=`date -d "$ltc_to_process_formatted $(date -d
"$find_timecode_formatted" +%s.%N) seconds ago" +%s.%N`
if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff"
then
printf "\non dateDiff $dateDiff , ltc_to_process_formatted
$ltc_to_process_formatted is later than or equal to find_timecode_formatted
$find_timecode_formatted \n"
else
q81TimeCodeInProgress=1
printf "\non dateDiff $dateDiff , ltc_to_process_formatted
$ltc_to_process_formatted is less than find_timecode_formatted
$find_timecode_formatted \n"
printf "\nadding video to file list to send to all.mov: "
printf "file %s\n" "${q81VideoFile}"
printf "file %s\n" "${q81VideoFile}" >> all.txt
printf "%s\n" "${q81VideoFile}.wav" >> debug.txt
continue
fi
printf "\nfound find_timecode: "
printf " %s" "${find_timecode}"
printf "\nin file: "
printf " %s" "${q81VideoFile}.wav"
printf "\nwill now process the q82 files looking for timecode later
than or equal to: "
printf " %s" "${find_timecode_formatted}"
printf "\nand timecode before or equal to: "
printf " %s" "${next_timecode_to_find_formatted}"
q81TimeCodeInProgress=0
saveQ81CurrentTimeCodeInProgress_to_find_formatted=$ltc_to_process_formatted
saveQ81CurrentTimeCodeInProgressAlreadyProcessed=0
saveQ82CurrentTimeCodeInProgressAlreadyProcessed=0
while read f; do
# skip auto generated version line
if [[ $f =~ .*ffconcat.* ]]; then
continue
fi
q82VideoFile=`echo $f | cut -d " " -f2-`;
printf "%s\n" "${q82VideoFile}" >> q82_processed.txt
# convert Q82 video segments to wav,
# so that ltcdump can find the timecode
if [[ ! -f "${q82VideoFile}.wav" ]]; then
ffmpeg -v error -nostdin -i $q82VideoFile -vn -c:a pcm_s16le
-ar 48000 -ac 2 "${q82VideoFile}.wav"
fi
validate "${q82VideoFile}.wav"
if [[ $found_stop_time -eq 1 ]]; then
printf "\nq82 video is after stop time, executing break\n"
break
fi
if [[ $saveQ81CurrentTimeCodeInProgressAlreadyProcessed -eq 0 &&
$q82TimeCodeInProgress -eq 0 && $isLTCDumpOutputValid -eq 1 ]]; then
saveQ81CurrentTimeCodeInProgressAlreadyProcessed=1
dateDiff=0
dateDiff=`date -d
"$saveQ81CurrentTimeCodeInProgress_to_find_formatted $(date -d
"$ltc_to_process_formatted" +%s.%N) seconds ago" +%s.%N`
if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0"
then
printf "\nadding q81 file in q82 loop on dateDiff $dateDiff
, $saveQ81CurrentTimeCodeInProgress_to_find_formatted is before or equal to
$ltc_to_process_formatted"
printf "file %s\n" "${q81VideoFile}" >> all.txt
printf "%s\n" "${q81VideoFile}.wav" >> debug.txt
else
printf "\nin q82 loop no action taken on q81 video file
"${q81VideoFile}" , dateDiff $dateDiff ,
$saveQ81CurrentTimeCodeInProgress_to_find_formatted greater than
$ltc_to_process_formatted"
fi
fi
if [[ $isLTCDumpOutputValid -eq 0 && $q82TimeCodeInProgress -eq 1
]]; then
printf "\nfound LTC error while processing files in range, just
accept it"
printf "\nadding video to q82 file processed list"
printf "file %s\n" "${q82VideoFile}" >> all.txt
printf "%s\n" "${q82VideoFile}.wav" >> debug.txt
continue
fi
if [[ $isLTCDumpOutputValid -eq 0 ]]; then
printf "\nFound ltc error outside time range while processing a
q82 file out of range, reject all q82 videos when this happens"
printf "\nskipping file: "
printf " %s" "${q82VideoFile}"
continue
fi
dateDiff=0
dateDiff=`date -d "$ltc_to_process_formatted $(date -d
"$find_timecode_formatted" +%s.%N) seconds ago" +%s.%N`
if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff"
then
printf "\nproceeding on file "${q82VideoFile}" , dateDiff
$dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is later or
equal to find_timecode_formatted $find_timecode_formatted"
else
printf "\nskipping file "${q82VideoFile}" on dateDiff $dateDiff
, ltc_to_process_formatted $ltc_to_process_formatted is before
find_timecode_formatted $find_timecode_formatted"
continue
fi
dateDiff=0
dateDiff=`date -d "$ltc_to_process_formatted $(date -d
"$next_timecode_to_find_formatted" +%s.%N) seconds ago" +%s.%N`
if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0"
then
printf "\nproceeding on dateDiff $dateDiff ,
$ltc_to_process_formatted is before or equal to
$next_timecode_to_find_formatted"
else
printf "\nexecuting break on file "${q82VideoFile}" , dateDiff
$dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is greater
than next_timecode_to_find $next_timecode_to_find"
break
fi
q82TimeCodeInProgress=1
printf "\nfound ltc in range"
printf "\nadding video to q82 file processed list"
printf "file %s\n" "${q82VideoFile}" >> all.txt
printf "%s\n" "${q82VideoFile}.wav" >> debug.txt
done < q82t.ffcat
q82TimeCodeInProgress=0
if [[ ! -f "q82_processed.txt" ]]; then
printf "\n\nCould not find file: q82_processed.txt , no timecode in
q82 files found between $timecode_to_find and $next_timecode_to_find\n"
exit
else
printf "\nq82 loop completed, removing files already processed from
the queue"
grep -F -v -f q82_processed.txt q82t.ffcat >
q82t_files_not_processed.txt
mv q82t_files_not_processed.txt q82t.ffcat
fi
# reset everything for next loop
let epoch_secs_rejectBeforeTime=$epoch_secs_next_timecode_to_find+1
reject_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000
$epoch_secs_rejectBeforeTime seconds" | cut -b1-11 | sed 's/\(.*\):/\1./'
`
rejectBeforeTime_formatted="$timecode_date ${reject_timecode}"
hasRejectBeforeTime=1
increment=$(( $timeDivisionInSeconds * 2 ))
let epoch_secs_timecode_to_find=$epoch_secs_timecode_to_find+$increment
find_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000
$epoch_secs_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' `
find_timecode_formatted="$timecode_date ${find_timecode}"
let
epoch_secs_next_timecode_to_find=$epoch_secs_next_timecode_to_find+$increment
next_timecode_to_find=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00
+0000 $epoch_secs_next_timecode_to_find seconds" | cut -b1-11 | sed
's/\(.*\):/\1./' `
next_timecode_to_find_formatted="$timecode_date ${next_timecode_to_find}"
printf "\nloop completed, timecode_to_find has been reset: "
printf " %s" "${find_timecode_formatted}"
printf "\nnext_timecode_to_find_formatted has been reset: "
printf " %s" "${next_timecode_to_find_formatted}"
printf "\nrejectBeforeTime_formatted has been reset: "
printf " %s" "${rejectBeforeTime_formatted}"
done < q81t.ffcat
printf "\nExecuting: ffmpeg -y -v error -nostdin -f concat -fflags +genpts
-safe 0 -i all.txt -async 1 -vsync 1 -c:a pcm_s16le -c:v libx264 -refs 1
-x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k
-bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af
"aresample=async=1:min_hard_comp=0.100000:first_pts=0" -crf 0 all.mov"
ffmpeg -y -v error -nostdin -f concat -fflags +genpts -safe 0 -i all.txt
-async 1 -vsync 1 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0
-r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate
1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af
"aresample=async=1:min_hard_comp=0.100000:first_pts=0" -crf 0 all.mov
# debug
ffmpeg -y -v error -nostdin -i all.mov -vn -c:a pcm_s16le -ar 48000 -ac 2
all.wav
while read wavFile; do
ltc_out=`ltcdump $wavFile 2>/dev/null `
printf "### output from file %s\n\n" "${wavFile}" >> debug_out.txt
printf "%s\n\n" "${ltc_out}" >> debug_out.txt
done < debug.txt | sort -V
exit
More information about the ffmpeg-user
mailing list