[FFmpeg-devel] Add protocol for Android content providers

[Zhao Zhili]

> On Feb 27, 2024, at 15:17, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
> 
> On Sat, Feb 24, 2024 at 12:29:24PM +0100, Matthieu Bouron wrote:
>> On Thu, Feb 15, 2024 at 10:13:03AM +0100, Matthieu Bouron wrote:
>>> Le jeu. 15 févr. 2024, 9:46 AM, Zhao Zhili <quinkblack at foxmail.com> a
>>> écrit :
>>> 
>>>> 
>>>>> 在 2024年2月15日，下午3:57，Matthieu Bouron <matthieu.bouron at gmail.com> 写道：
>>>>> 
>>>>> On Thu, Feb 15, 2024 at 12:13:59PM +0800, Zhao Zhili wrote:
>>>>>> 
>>>>>> 
>>>>>>>> On Feb 14, 2024, at 06:50, Matthieu Bouron <matthieu.bouron at gmail.com>
>>>> wrote:
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> On Android, content providers are used for accessing files through
>>>> shared
>>>>>>> mechanisms. One typical case would be an app willing to open a video
>>>> from
>>>>>>> Google Photos, gallery apps, TikTok, Instagram or some other providers.
>>>>>>> A content URI looks something like "content://authority/path/id", see:
>>>>>>> https://developer.android.com/reference/android/content/ContentUris
>>>>>>> 
>>>> https://developer.android.com/guide/topics/providers/content-provider-basics
>>>>>>> 
>>>>>>> It can currently be somehow managed through clumsy means such as using
>>>> a "fd:"
>>>>>>> filename and crafting a special AVOption, which also has the drawback
>>>> of
>>>>>>> requiring the third party to carry around opened file descriptors
>>>> (with the
>>>>>>> multiple opened file limitations implied). Custom AVIOContexts are
>>>> also an
>>>>>> 
>>>>>> File descriptor is a general abstraction layer, it target more
>>>> platforms than
>>>>>> Android specific content provider. Android provided getFd() API since
>>>> API
>>>>>> level 12, I guess that’s the default method to deal with content
>>>> provider in
>>>>>> native code. It’s a few lines of code to get native fd in Java, but
>>>> dozens of code
>>>>>> in C with JNI, which is what this patchset done.
>>>>>> 
>>>>>> For multiple opened file limitations issue, they can close the file
>>>> descriptor after
>>>>>> open. It’s unlikely to reach the limit in normal case without leak.
>>>>>> 
>>>>>> I’m OK to provide this android_content_protocol helper if user requests.
>>>>> 
>>>>> I've been doing this kind of work for 3/4 users (including myself) at
>>>> this
>>>>> point and have to do it another time, this is what motivated me to
>>>> propose
>>>>> this patchset.
>>>>> 
>>>>>> 
>>>>>>> option. Both options will have to deal with the JNI though and end
>>>> users will
>>>>>>> have to re-implement the same exact thing.
>>>>>> 
>>>>>> User still need to deal with JNI with the new android_content_protocol,
>>>> more or
>>>>>> less, it’s unavoidable.
>>>>> 
>>>>> The advantage I see of using this protocol is that the user only need to
>>>>> call av_jni_set_jvm() + av_jni_set_android_app_ctx() at the start of the
>>>>> application and FFmpeg will handle the content-uri transparently. This is
>>>>> especially helpful if the Android application rely on multiple libraries
>>>>> that in turn rely on FFmpeg to read medias.
>>>> 
>>>> The url still need to be passed from Java to C via JNI, it’s not much
>>>> different compared to pass fd.
>>>> 
>>> 
>>> It's not that much different I agree. But let's say you have a rendering
>>> engine (in C) where you need to pass hundreds of media (from the user) to
>>> render a scene, each media is used at different time during the rendering.
>>> And Ffmpeg is not a direct dependency and can be called from different
>>> libraries/places used by the rendering engine. Calling
>>> av_jni_set_android_app_ctx() and you're done, you can pass the content URI
>>> to the engine (passing fd at this stage is not an option imho). You still
>>> need to convert the uri from java string to c before calling the c code,
>>> but it's a direct translation which is typically part of a binding.
>>> 
>>> 
>>> 
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> This patchset addresses this by adding a content provider protocol,
>>>> which has
>>>>>>> an API fairly similar to fopen. Android 11 appears to provide something
>>>>>>> transparent within fopen(), but FFmpeg doesn't use it in the file
>>>> protocol, and
>>>>>>> Android < 11 are still widely used.
>>>>>>> 
>>>>>>> The first part move the JNI infrastructure from avcodec to avutil (it
>>>> remains
>>>>>>> internally shared, there is little user implication),
>>>>>> 
>>>>>> OK. JNI infrastructure should belong to avutil at the first place, so
>>>> hwcontext_mediacodec
>>>>>> and so on can use it. Unfortunately for those new avpriv_.
>>>>> 
>>>>> What do you mean by "Unfortunately" ? Would you like to make the JNI API
>>>>> public ?
>>>> 
>>>> I think it’s our target to reduce the number of avpriv API, not increase
>>>> it. Does duplicate the compile unit work in this case so we don’t need to
>>>> export the symbols?
>>>> 
>>> 
>>> Directly including ffjni.c from libavformat/file.c works. We still need to
>>> pass the application context though (could be added to avcodec/jni.h)
>> 
>> So what would be the preferred way forward ? including libavformat/file.c or
>> migrating the code to avutil (avpriv_*) ?
> 
> Ping (sorry to ping this early, I'd like to not miss the 7.0 window,
> especially if we choose the avpriv_ route).

I prefer the solution to dup compile unit and use as less avpriv_ as possible.

> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".