[FFmpeg-devel] [PATCH 14/14] videotoolbox: remove opengl compatibility attribute

[Xiaolei Yu]

On 12/16/2017 02:07 PM, Wang Bin wrote:
> 2017-12-16 2:52 GMT+08:00 wm4 <nfxjfg at googlemail.com>:
>> On Fri, 15 Dec 2017 15:02:44 +0800
>> wbsecg1 at gmail.com wrote:
>>
>>> From: wang-bin <wbsecg1 at gmail.com>
>>>
>>> 1. a cvpixelbuffer backed by iosurface can always be converted to an opengl texture, using CGLTexImageIOSurface2D for macOS, and undocumented api texImageIOSurface(which is internally used by public api CVOpenGLESTextureCacheCreateTextureFromImage) for iOS4.0+.
>>> 2. enabling the attribute can slow down decoding speed a lot. I tested many video clips on my macbook air. for example: ffmpeg -ss 00:00:00 -t 00:03:00 -hwaccel videotoolbox -an -i big_buck_bunny_1080p_h264.mov -f null ->/dev/null, result with the attribute
>>> enabled: frame= 2082 fps= 85 q=-0.0 Lsize=N/A time=00:03:00.00 bitrate=N/A speed=7.34x
>>> disabled: frame= 2031 fps=104 q=-0.0 Lsize=N/A time=00:03:00.00 bitrate=N/A speed=9.22x
>>> ---
>>>  libavcodec/videotoolbox.c | 5 -----
>>>  1 file changed, 5 deletions(-)
>>>
>>> diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c
>>> index 9d2f0afa20..24631684d7 100644
>>> --- a/libavcodec/videotoolbox.c
>>> +++ b/libavcodec/videotoolbox.c
>>> @@ -664,11 +664,6 @@ static CFDictionaryRef videotoolbox_buffer_attributes_create(int width,
>>>      CFDictionarySetValue(buffer_attributes, kCVPixelBufferIOSurfacePropertiesKey, io_surface_properties);
>>>      CFDictionarySetValue(buffer_attributes, kCVPixelBufferWidthKey, w);
>>>      CFDictionarySetValue(buffer_attributes, kCVPixelBufferHeightKey, h);
>>> -#if TARGET_OS_IPHONE
>>> -    CFDictionarySetValue(buffer_attributes, kCVPixelBufferOpenGLESCompatibilityKey, kCFBooleanTrue);
>>> -#else
>>> -    CFDictionarySetValue(buffer_attributes, kCVPixelBufferIOSurfaceOpenGLTextureCompatibilityKey, kCFBooleanTrue);
>>> -#endif
>>>
>>>      CFRelease(io_surface_properties);
>>>      CFRelease(cv_pix_fmt);
>>
>> Does this have a negative effect on compatibility or performance? (In
>> both cases I'm asking about the case when actually using GL rendering.)
> 
> Disabling the attribute improves performance in my tests. I can not
> find any document about these keys. What i know is the decoded
> cvpixelbuffer is backed by iosurface, and the api to create texture
> from iosurface is available since macOS10.6

Maybe you can add a flag to make them optional?
They are documented at:
https://developer.apple.com/documentation/corevideo/kcvpixelbufferopenglescompatibilitykey
https://developer.apple.com/documentation/corevideo/kcvpixelbufferiosurfaceopengltexturecompatibilitykey

Things may still work without them but I would like to follow the documentation whenever possible. And I think iOS DOES require the flag for those buffers to be consumed through GLES.

Pure speculation here. GPU texture units usually require special memory layouts that may be suboptimal for the decoder. When these constraints are not met either the driver has to perform the conversion or you pay the penalty at access time.


> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>