[FFmpeg-devel] [RFC]] swscale modernization proposal

[Michael Niedermayer]

On Sat, Jun 22, 2024 at 05:10:28PM +0200, Niklas Haas wrote:
> On Sat, 22 Jun 2024 15:23:22 +0100 Andrew Sayers <ffmpeg-devel at pileofstuff.org> wrote:
> > On Sat, Jun 22, 2024 at 03:13:34PM +0200, Niklas Haas wrote:
> > [...]
> > > 
> > > ## Comments / feedback?
> > > 
> > > Does the above approach seem reasonable? How do people feel about introducing
> > > a new API vs. trying to hammer the existing API into the shape I want it to be?
> > > 
> > > I've attached an example of what <avscale.h> could end up looking like. If
> > > there is broad agreement on this design, I will move on to an implementation.
> > 
> > API users seem to have difficulty with this type of big change[[1],
> > and doing the interface before the implementation means there's less
> > reason for developers to switch while you're still looking for feedback.
> > 
> > What's the plan to bring them along?
> 
> Since SwsContext is entirely internal, we can continue providing the
> current API on top of whatever internal abstractions we arrive at. As
> a trivial example, sws_scale() can construct a temporary AVFrame based
> on the provided information, and simply pass that to avscale_frame(). So
> I don't think legacy API users are at risk, or pressure to switch,
> unless they want access to *new* functionality.
> 
> For that, the harder step is moving from sws_scale() to
> sws_scale_frame(). This is something API users can *already* do. By
> contrast, moving from sws_scale_frame() to avscale_frame() should
> hopefully be simple, since it just requires making sure the AVFrame is
> correctly tagged. Usually, the flow is in the opposite direction - users
> receive a correctly tagged AVFrame and manually forward this information
> to the SwsContext. So, most of the time, moving to a fully AVFrame-based
> API will result in deleting code, rather than adding it.
> 
> If we wanted to maximize the transition comfort, we should consider
> re-using the sws_scale_frame() entrypoint directly. The main reason I am
> hesitant to do this is because sws_scale_frame() is currently tied to
> the stateful configuration of SwsContext, and mostly ignores the AVFrame
> metadata. While changing that is possible, it possibly presents a bigger
> API break than simply introducing a new function.

I agree we should keep using the same swscale.h header. It matches the library
name thats installed (thats also what the user expects and what (s)he is used to),
and its what users #include today.
Also its not a audio? scaler so the A is confusing.

And sws_scale_frame() should be used obviously if thats as you say does
"maximize the transition comfort"

Maybe simply adding an option for the library user to set the behavior
(favour AVFrame properties vs initial properties)
And then eventually deprecate and phase out the initial ones

The big advantage here is that we capture all users, noone stays on the old
API. And the transition is also simpler, if its just a flag to flip for someone
to try the new fully stateless system.

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Complexity theory is the science of finding the exact solution to an
approximation. Benchmarking OTOH is finding an approximation of the exact
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240622/2190e45a/attachment.sig>