[FFmpeg-devel] [PATCH] hwcontext_videotoolbox: add vt_device_derive

[ChenLiucheng]

Well this does make sense. 

If only one type of hardware device is in use, then `-init_hw_device` will suffice. However, if we use hardware filters with different device types, such as OpenCL, and we want to switch to a VideoToolbox filter later in the chain, we will have a issue. Since there is no `hwmap` between OpenCL and VideoToolbox, we need to perform a `hwdownload` followed by a `hwupload`. If the `derive_device` option cannot be used with `hwupload` due to `device_derive` not being implemented, the filter will struggle to find the correct device for uploading. It may attempt to upload to the OpenCL device without `device_derive` set if it comes after an OpenCL filter.

> On Feb 23, 2024, at 03:28, Mark Thompson <sw at jkqxz.net> wrote:
> 
> On 22/02/2024 18:46, gnattu via ffmpeg-devel wrote:
>> There is no device context to be setup, nor devices to be
>> selected with VideoToolbox. Just a simple return would allow
>> us to use derived device in filters like
>> `hwupload=derive_device=videotoolbox`
>> Signed-off-by: Gnattu OC <gnattuoc at me.com>
>> ---
>>  libavutil/hwcontext_videotoolbox.c | 9 +++++++++
>>  1 file changed, 9 insertions(+)
>> diff --git a/libavutil/hwcontext_videotoolbox.c b/libavutil/hwcontext_videotoolbox.c
>> index fe469dc161..d13199eca7 100644
>> --- a/libavutil/hwcontext_videotoolbox.c
>> +++ b/libavutil/hwcontext_videotoolbox.c
>> @@ -759,6 +759,14 @@ static int vt_device_create(AVHWDeviceContext *ctx, const char *device,
>>      return 0;
>>  }
>>  +static int vt_device_derive(AVHWDeviceContext *device_ctx,
>> +                            AVHWDeviceContext *src_ctx, AVDictionary *opts,
>> +                            int flags)
>> +{
>> +    // There is no context to be setup with VT, just return.
>> +    return 0;
>> +}
>> +
>>  const HWContextType ff_hwcontext_type_videotoolbox = {
>>      .type                 = AV_HWDEVICE_TYPE_VIDEOTOOLBOX,
>>      .name                 = "videotoolbox",
>> @@ -766,6 +774,7 @@ const HWContextType ff_hwcontext_type_videotoolbox = {
>>      .frames_priv_size     = sizeof(VTFramesContext),
>>        .device_create        = vt_device_create,
>> +    .device_derive        = vt_device_derive,
>>      .frames_hwctx_size    = sizeof(AVVTFramesContext),
>>      .frames_init          = vt_frames_init,
>>      .frames_get_buffer    = vt_get_buffer,
> 
> This derivation behaviour doesn't make any sense inside libavutil.  Features which are only for the ffmpeg utility should be implemented inside the ffmpeg utility.
> 
> (Also, try -init_hw_device.)
> 
> Thanks,
> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org <mailto:ffmpeg-devel-request at ffmpeg.org> with subject "unsubscribe".