[FFmpeg-user] Is there something about inputting raw frames

Mark Filipak (ffmpeg) markfilipak at bog.us
Fri Feb 12 10:33:01 EET 2021


On 02/12/2021 02:48 AM, pdr0 wrote:
> Mark Filipak (ffmpeg) wrote
>> On 02/12/2021 02:28 AM, pdr0 wrote:
>>> Mark Filipak (ffmpeg) wrote
>>>> On 02/12/2021 01:27 AM, pdr0 wrote:
>>>>> Mark Filipak (ffmpeg) wrote
>>>>>> Is there something about inputting raw frames that I don't know?
>>>>>>
>>>>>> I'm using 'vspipe' to pipe raw frames to 'ffmpeg -i pipe:'.
>>>>>> The vapoursynth script, 'Mark's.vpy', is known good.
>>>>>> The output of vapoursynth is known good.
>>>>>> I've tried to be careful to retain valid PTSs, but apparently have
>>>>>> failed.
>>>>>> The output should be around 1200 frames, but 364 frames are dropped.
>>>>>> I've frame stepped through the target, 'Mark's_script_6.mkv', and the
>>>>>> frames that are there are in
>>>>>> the correct order.
>>>>>> The only thing I can guess is that ffmpeg handles 48/1.001fps raw
>>>>>> video
>>>>>> frames in such a way that
>>>>>> PTS is not valid or can't be changed with 'setpts=fps=60000/1001'.
>>>>>> Can anyone see an error. Or, lacking an error, does anyone know of a
>>>>>> workaround?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> Mark's_script_6.cmd
>>>>>> =====
>>>>>> ECHO from vapoursynth import core>Mark's.vpy
>>>>>> ECHO video =
>>>>>> core.ffms2.Source(source='Mark\'s_source.mkv')>>Mark's.vpy
>>>>>> ECHO import havsfunc as havsfunc>>Mark's.vpy
>>>>>> ECHO video = havsfunc.InterFrame(video, Preset="medium",
>>>>>> Tuning="smooth",
>>>>>> InputType="2D",
>>>>>> NewNum=48000, NewDen=1001, GPU=True)>>Mark's.vpy
>>>>>> ECHO video.set_output()>>Mark's.vpy
>>>>>> vspipe --y4m Mark's.vpy - | ffmpeg -thread_queue_size 2048 -i pipe:
>>>>>> -filter_complex
>>>>>> "setpts=N*1001/60000/TB, split[1][2], [1]shuffleframes=0 1 2 3 3,
>>>>>> select=not(eq(mod(n\,5)\,4))[3],
>>>>>> [2]tblend=all_expr='if(eq(mod(X,2),mod(Y,2)),TOP,BOTTOM)',
>>>>>> shuffleframes=0
>>>>>> 1 2 2 3,
>>>>>> select=eq(mod(n\,5)\,4)[4], [3][4]interleave" -i Mark's_source.mkv
>>>>>> -map
>>>>>> 0:v -map 1:a -codec:v
>>>>>> libx265 -x265-params "crf=16:qcomp=0.60" -codec:a copy -codec:s copy
>>>>>> Mark's_script_6.mkv -y
>>>>>
>>>>>
>>>>> Are you trying to keep the same frames from vapoursynth output node,
>>>>> but
>>>>> assign 60000/1001 fps and timestamps instead of 48000/1001 ?
>>>>> (effectively
>>>>> making it a speed up)
>>>>>
>>>>> If so, the workaround is : add after the Interframe line
>>>>>
>>>>> video = core.std.AssumeFPS(video, fpsnum=60000, fpsden=1001)
>>>>
>>>> After your suggested addition to the python script, Mark's.vpy,
>>>> With '-filter_complex "setpts=N*1001/60000/TB, split[1][2]...' there are
>>>> 335 drops.
>>>> With '-filter_complex "split[1][2]...' there are 190 drops.
>>>
>>> The workaround is correct for the PTS
>>>
>>>    AssumeFPS is used to change the frame rate (and timestamps) without
>>> changing the frame count. It just assigns a framerate (and their PTS). So
>>> you would use that instead of setpts
>>>
>>> There are no frame drops or additions from vapoursynth. The framecount,
>>> framerate and PTS are correct at that point. You can verify this by
>>> encoding
>>> the vpy script directly without other filters.
>>>
>>> So this implies the some drops are from setpts, and some other drops are
>>> from some of your other filters
>>
>> Well, I previously removed the 'setpts' directives with no change. Also, I
>> previously tested with no
>> ffmpeg filters at all and got the expected sped up video. I honestly can't
>> see anything else to
>> discover. But I'll start stripping filters one by one and hack a solution
>> (or make a discovery). Of
>> course, the 'shuffleframes' directives are most suspect, but I've used
>> 'shuffleframes' in the past
>> and succeeded.
>>
>> Thanks for your help. It was instrumental. The rest is up to me.
> 
> 
> 
> 
> I realize this is ffmpeg-user board, but why not do some of the video
> processing in vapoursynth ? You're already using it for part of it.  But I'm
> not sure what you're trying to do ?
> 
> It looks like you're taking the 48000/1001 interpolated frame doubled output
> and adding a "BC~C.bc~c" frame to every group of 4, to make up the
> 60000/1001?
> 
> What is a "BC~C.bc~c" frame ? Is it that checkerboard blend?
> http://ffmpeg.org/pipermail/ffmpeg-user/2021-February/051852.html
> 
> Either way, cadence wise that's going to be worse in terms of smoothness
> then an optical flow retimed 60000/1001 . (Some people would argue it's
> worse period, you're retiming it and making it look like a soap opera...)

You know, I think that "soap opera" opinion is entirely bogus. In a movie theater, projectors are 
triple shuttered. That essentially brings the frame rate up to 72fps with picture rate of 24pps (or, 
if you include the times when the shutter is closed, 144fps). When some people see on a TV what they 
would see in a cinema, they say it looks like a soap opera. It's not what they're used to seeing on 
a TV. I think 60fps on a 60Hz TV looks much better, that it looks like reality. If you've been 
following what I've been doing, you'll know that's been my objective all along. I'd hoped that 
minterpolate would do it, but minterpolate makes too many errors. svpflow does a much better job and 
it does it via GPU (so transcoding goes from 4 days to 14 hours). I'm pretty confident that going to 
48fps (instead of 60fps) and then adding a modulo-4 frame will speed up the transcode by about 40% 
(to 8-1/2 hours -- an overnight job!).

> Are you actually interested in workarounds and getting the job done, or just
> how to do this in ffmpeg?

Well, I guess I just want to get the job done. The linchpin is the added frame. What I want to do is 
create a modulo-4, 1/60th second gap in the frames coming out of VapourSynth and filling it with a 
checkerboard blend of the frames on either side of the gap -- essentially a blended decombing -- 
with PTSs set to give 60/1.001fps. I realize that will produce a slight judder (1 frame in every 5 
frames, but based on my experiments with minterplolate, that judder is *almost* imperceptible. If 
shuffleframes proves to be the problem, I'll do a 3322telecine and checkerboard blend the combed 
frame. I'll get where I want to go eventually. All the guesswork & discovery regarding how ffmpeg 
filters work is just awfully tedious.

> If you just want it done, this is easier in avisynth because of the
> InterleaveEvery function;
> http://avisynth.nl/index.php/ApplyEvery#InterleaveEvery

Oh, my. Avisynth? 'InterleaveEvery', eh? That doesn't sound like what I want, but I'll check it out. 
Thanks.

-- 
The U.S. Senate and House should follow the ancient Roman model.
Their Princeps Senatus and Tribunus Plebis had their own armies.


More information about the ffmpeg-user mailing list