[FFmpeg-devel] [PATCH] avdevice/avdevice: Deprecate AVDevice Capabilities API

[Nicolas George]

Mark Thompson (12021-01-25):
> Merging the libraries (in source form, orthogonal to merging the
> binaries) only makes sense if we are going to continue using the
> libavformat internals, and that is exactly the thing we are trying to
> get rid of.

"We"?

It seems to me that many people here have an agenda about libavdevice
while they have no reason to.

> If we replace the libavdevice API with a proper one oriented towards
> devices rather than being hacked on to the libavformat API then we
> actively don't want that merger.
> 
> So, what do people actually want here?  The situation as I see it is
> (please correct me on anything you feel is in error here):
> 
> * The features which libavdevice offers are useful to a lot of people.

Thank you for acknowledging it.

> * Many of those are using it via the ffmpeg utility, but not all.
> * The libavdevice API is the libavformat API because it was originally
> split out from libavformat, and it has the nice property that devices
> and files end up being interchangable in some contexts.
> * The libavdevice API, being the libavformat API for files, is not
> particularly well-suited in other contexts, because devices may not
> have the same properties as files.
> * Some odd things like the completely-unused capabilities API and the
> almost-never-used message API are hacked on top of that to try to
> avoid some libavformat issues, but are not actually useful to anyone
> (hence the lack of use).
> * To implement devices as AVInputFormat/AVOutputFormat instances,
> libavdevice currently needs access to the internals of libavformat.
> * Many developers want to get rid of that dependency on libavformat
> internals, because it creates a corresponding ugliness on the
> libavformat side which has to leave those parts exposed in an
> ABI-constrained way.
> 
> What might we do about this:
> 
> * Delete libavdevice, with a statement that its capabilities are
> outside the scope of ffmpeg.  Obviously this isn't sensible, the
> ffmpeg utility would then need to depend on some external library to
> offer the same features it does now.
> * Merge libavdevice into the ffmpeg utility.  Since it is the primary
> user this would neatly sidestep the previous objection, but now
> external users are in the same boat so again unacceptable.

One of the main reasons the fork failed while original ffmpeg kept going
was that while ffmpeg has a policy of being useful, the fork had an
unspoken rule about code purity that lead to removing many useful
features. IIRC, Anton was one of the most vocal proponent of this
harmful unspoken rule.

I find worrisome to see the same attitude start to appear here. We need
to be very careful.

Also, if the project actually starts removing features for purity, like
some people discussed here, I'll stop bothering about this dying
project.

> * Merge libavdevice into libavformat.  This works to eliminate the
> dependency problem, but it only really makes sense if we accept that
> libavdevice is finished and will never want new API which isn't
> repurposed libavformat.
> * Make a new libavdevice-specific API and switch to that.  The
> eliminates the dependency problem, but there is a nasty transition
> period and it would break the interchangability of the two libraries.
> It also would be the most work, but leave libavdevice in the best
> state for doing more things with in future.

One of the important features of libavdevice, as you mentioned earlier,
is that devices can be a drop-in replacement for demuxers and muxers. An
application that is not designed for devices can be made to use them,
and that is immensely convenient. It does not work always, but it work
often enough to be, I repeat, immensely useful.

If a solution does not have this feature, we can just skip it.

> * Nothing.  People will continue to be annoyed about the problems above.

Duh. People often are annoyed at the wrong things, let them be annoyed.
I have had proof in the past that some of the annoyance on this subject
was actually a deep misunderstanding of how linking works.

There is another option:

* Stop imposing ourselves constraints that bring absolutely no benefit.
Having internals is useful for a library because we can change them
without bothering about compatibility: if we change the internals, we
can change all places that use them at the same time, and everything
keeps working.

It is a good hygiene that the fftools are written to use no internals,
because that proves the public API is complete enough to build powerful
tools.

But for the libraries, it makes no sense to have this constraint. What
benefit is there in it? Nobody has been able to answer this question,
because there are none. Libraries should be able to use each other
internals (typical example: sharing header decoding and format-specific
utility functions between lavc and lavf).

> Opinions?
> 
> Personally, I think the right option here is to make a new
> libavdevice-specific API and cut the connection to libavformat
> entirely, and I would be prepared to do some work towards that goal.
> Exactly what form it would take is unclear, but I would be happy to
> talk about that further if people agree that it is the right approach.

One of the very important things you missed:

A lot of devices would also be very useful as libavfilter sources and
sinks. It allows automatic format negotiation, direct rendering and many
other benefits.

But it cannot work for now, because the API to run a filter graph only
works with buffer sinks, no other kinds of sinks.

Therefore, before considering enhancing the API of libavdevice, we need
a proper API tu run a filter graph.

And since we want inter-filters parallelism, a feature that has been
wanted for a long time, for that we need a proper event loop and
parallel task scheduler. Which brings us back to this mail, on which I
intend to follow-up soon:
https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/274172.html

Yes, libavdevice's API is far from perfect, but it is part of a much
wider problem: everything in FFmpeg that is not directly codecs and
formats, and secondarily filters, has been let to slowly rot, because
"we don't need this", "it's not our core mission", "it's unusual",
whatever.

Continue like that, and FFmpeg will become irrelevant. Or change
completely, embrace originality, create the necessary infrastructure,
and FFmpeg can be again a leader in efficient audio-video programming.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210126/b14dd170/attachment.sig>