[FFmpeg-devel] Add protocol for Android content providers

[Matthieu Bouron]

On Sat, Feb 24, 2024 at 12:29:24PM +0100, Matthieu Bouron wrote:
> On Thu, Feb 15, 2024 at 10:13:03AM +0100, Matthieu Bouron wrote:
> > Le jeu. 15 févr. 2024, 9:46 AM, Zhao Zhili <quinkblack at foxmail.com> a
> > écrit :
> > 
> > >
> > > > 在 2024年2月15日，下午3:57，Matthieu Bouron <matthieu.bouron at gmail.com> 写道：
> > > >
> > > > On Thu, Feb 15, 2024 at 12:13:59PM +0800, Zhao Zhili wrote:
> > > >>
> > > >>
> > > >>>> On Feb 14, 2024, at 06:50, Matthieu Bouron <matthieu.bouron at gmail.com>
> > > wrote:
> > > >>>
> > > >>> Hi,
> > > >>>
> > > >>> On Android, content providers are used for accessing files through
> > > shared
> > > >>> mechanisms. One typical case would be an app willing to open a video
> > > from
> > > >>> Google Photos, gallery apps, TikTok, Instagram or some other providers.
> > > >>> A content URI looks something like "content://authority/path/id", see:
> > > >>> https://developer.android.com/reference/android/content/ContentUris
> > > >>>
> > > https://developer.android.com/guide/topics/providers/content-provider-basics
> > > >>>
> > > >>> It can currently be somehow managed through clumsy means such as using
> > > a "fd:"
> > > >>> filename and crafting a special AVOption, which also has the drawback
> > > of
> > > >>> requiring the third party to carry around opened file descriptors
> > > (with the
> > > >>> multiple opened file limitations implied). Custom AVIOContexts are
> > > also an
> > > >>
> > > >> File descriptor is a general abstraction layer, it target more
> > > platforms than
> > > >> Android specific content provider. Android provided getFd() API since
> > > API
> > > >> level 12, I guess that’s the default method to deal with content
> > > provider in
> > > >> native code. It’s a few lines of code to get native fd in Java, but
> > > dozens of code
> > > >> in C with JNI, which is what this patchset done.
> > > >>
> > > >> For multiple opened file limitations issue, they can close the file
> > > descriptor after
> > > >> open. It’s unlikely to reach the limit in normal case without leak.
> > > >>
> > > >> I’m OK to provide this android_content_protocol helper if user requests.
> > > >
> > > > I've been doing this kind of work for 3/4 users (including myself) at
> > > this
> > > > point and have to do it another time, this is what motivated me to
> > > propose
> > > > this patchset.
> > > >
> > > >>
> > > >>> option. Both options will have to deal with the JNI though and end
> > > users will
> > > >>> have to re-implement the same exact thing.
> > > >>
> > > >> User still need to deal with JNI with the new android_content_protocol,
> > > more or
> > > >> less, it’s unavoidable.
> > > >
> > > > The advantage I see of using this protocol is that the user only need to
> > > > call av_jni_set_jvm() + av_jni_set_android_app_ctx() at the start of the
> > > > application and FFmpeg will handle the content-uri transparently. This is
> > > > especially helpful if the Android application rely on multiple libraries
> > > > that in turn rely on FFmpeg to read medias.
> > >
> > > The url still need to be passed from Java to C via JNI, it’s not much
> > > different compared to pass fd.
> > >
> > 
> > It's not that much different I agree. But let's say you have a rendering
> > engine (in C) where you need to pass hundreds of media (from the user) to
> > render a scene, each media is used at different time during the rendering.
> > And Ffmpeg is not a direct dependency and can be called from different
> > libraries/places used by the rendering engine. Calling
> > av_jni_set_android_app_ctx() and you're done, you can pass the content URI
> > to the engine (passing fd at this stage is not an option imho). You still
> > need to convert the uri from java string to c before calling the c code,
> > but it's a direct translation which is typically part of a binding.
> > 
> > 
> > 
> > > >
> > > >>
> > > >>>
> > > >>> This patchset addresses this by adding a content provider protocol,
> > > which has
> > > >>> an API fairly similar to fopen. Android 11 appears to provide something
> > > >>> transparent within fopen(), but FFmpeg doesn't use it in the file
> > > protocol, and
> > > >>> Android < 11 are still widely used.
> > > >>>
> > > >>> The first part move the JNI infrastructure from avcodec to avutil (it
> > > remains
> > > >>> internally shared, there is little user implication),
> > > >>
> > > >> OK. JNI infrastructure should belong to avutil at the first place, so
> > > hwcontext_mediacodec
> > > >> and so on can use it. Unfortunately for those new avpriv_.
> > > >
> > > > What do you mean by "Unfortunately" ? Would you like to make the JNI API
> > > > public ?
> > >
> > > I think it’s our target to reduce the number of avpriv API, not increase
> > > it. Does duplicate the compile unit work in this case so we don’t need to
> > > export the symbols?
> > >
> > 
> > Directly including ffjni.c from libavformat/file.c works. We still need to
> > pass the application context though (could be added to avcodec/jni.h)
> 
> So what would be the preferred way forward ? including libavformat/file.c or
> migrating the code to avutil (avpriv_*) ?

Ping (sorry to ping this early, I'd like to not miss the 7.0 window,
especially if we choose the avpriv_ route).