[FFmpeg-user] Filter documentation -- PTSs

[Mark Filipak (ffmpeg)]

On 02/15/2021 01:56 AM, Chris Angelico wrote:
> On Mon, Feb 15, 2021 at 5:32 PM Mark Filipak (ffmpeg)
> <markfilipak at bog.us> wrote:
>>> frame->pts = (
>>>        (s->start_time == AV_NOPTS_VALUE) ? 0 : s->start_time) +
>>>        av_rescale(outlink->frame_count_in, s->ts_unit.num, s->ts_unit.den);
>>
>> I don't know what this: 'frame->pts', means. I have written a ton of assembly code for various
>> micros and people tell me, "It's easy, Mark. 'frame->pts' is a 'pointer' to memory as in assembly",
>> but I don't see the analog of a memory address register in it. If 'pts' is a pointer, then how can
>> 'frame' be written through that pointer and where does it get written? It's a mystery to me. Or is
>> 'frame' the pointer and 'pts' the memory location?
> 
> 'frame' is a pointer to a structure of some sort, and 'pts' is a named
> element within that structure. So you might have something like:
> 
> struct FrameyThingyWhatsit {
>      int foo;
>      int bar;
>      void *quux;
>      int pts;
>      const char *flurble;
> };
> 
> and then 'frame' would be a pointer to an in-memory structure of that
> type. When you refer to 'frame->pts', that means 'look at the spot in
> memory three words in from where frame points, and assign to that'.

Well, that makes perfect sense. Thank you, Chris! You've removed most of the mystery. One thing 
remains: What links the label "frame" to the structure "FrameyThingyWhatsit"? Is there a linker 
directive somewhere, and what would it look like?

Oh, one thing I should clarify. All the folks who tried to explain pointers to me were firmware 
writers. Perhaps that explains why they referred to memory, eh? Hahaha... never thought about it 
before until you showed me the real codewright's view.

> But I think Carl's point was that you can't simply look at an
> expression and decode it as algebra. To understand this line of code,
> you not only have to interpret the syntax of this exact line, but -
> and probably more importantly - you have to understand what the
> av_rescale function is doing.

Well, I think I know what 'av_rescale()' is doing (at least for the 'minterpolate=fps=48/1.001' & 
'telecine' filters). I ran some tests. Look:

24p-original timestamps.txt
     Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 
23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
frames.frame.0.best_effort_timestamp_time="0:00:00.000000"
frames.frame.479.best_effort_timestamp_time="0:00:19.978000"
ms/frame = 19978/479 = 41.708 = ms/frame(48p-minterpolate48)*2   <<==

48p-minterpolate48 timestamps.txt
     Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 
47.95 fps, 47.95 tbr, 1k tbn, 47.95 tbc (default)
frames.frame.0.best_effort_timestamp_time="0:00:00.000000"
frames.frame.955.best_effort_timestamp_time="0:00:19.916000"
ms/frame = 19916/955 = 20.854 = ms/frame(24p-original)/2   <<==

60p-minterpolate48,telecine timestamps.txt
     Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 
59.94 fps, 59.94 tbr, 1k tbn, 59.94 tbc (default)
frames.frame.0.best_effort_timestamp_time="0:00:00.000000"
frames.frame.1194.best_effort_timestamp_time="0:00:19.920000"
ms/frame = 19920/1194 = 16.683 = ms/frame(48p-minterpolate48)*4/5   <<==

See? PTS factors of '2' & '4/5'; just what one would expect. Those are the numbers that matter.

I imagine the rest of the code is involved in initialization of certain things that aren't already 
defined, and scaling for things that could be represented as floats but are initialized as scaled 
integers in order to improve the speed of the pipeline. Do those rationalizations sound reasonable 
to you?

So, to wrap up, for the documentation, what would be good is the changes to PTSs for ordinary (or 
usual) processing. Numbers like '2' & '4/5'. In the cases above, the numbers are simple, but in 
other cases, the numbers are hard to suss out and seem to be inconsistent. And regarding VFR, well, 
what would the numbers be? How would they be calculated?

If the VFR numbers can be expressed simply, that would be great. If expressing them requires some 
textual explanation, that would still be very helpful no matter how difficult.

All of the foregoing assumes that PTSs are key. Since no one has commented on that part of my OP, I 
assume that PTSs *are* key. If so, they should be expressed for each filter. I would help to do that 
by running tests, but I would need to know going in that there's a home in the documentation for my 
research. Otherwise, I'm not going to spend any more of my lifetime.

That's enough writing. Now it's bedtime (past bedtime). Thanks so much for your insights. You've 
really cleared up pointers -- simple!

Mark.

-- 
Any journey, no matter how long, is just a series of small steps.
"Government is the problem!" -- 1982 and onward.
"_______ is the enemy of the people!" -- 2016 and onward.
"You have to fight like hell or you're not going to have a country!" -- Jan 6, 2021.
It isn't the distance that's important, it's the direction.