[FFmpeg-devel] [PATCH 1/4] kmsgrab: Fix 32-bit RGB DRM format definitions

[Mark Thompson]

On 15/09/17 22:49, Carl Eugen Hoyos wrote:
> 2017-09-15 23:44 GMT+02:00 Mark Thompson <sw at jkqxz.net>:
>> On 15/09/17 22:40, Carl Eugen Hoyos wrote:
>>> 2017-09-15 22:51 GMT+02:00 Mark Thompson <sw at jkqxz.net>:
>>>> The 32-bit DRM formats are defined in terms of little-endian words, so
>>>> 32-bit RGB formats like XRGB actually have the elements in the opposite
>>>> order in memory to the order they are in the name.
>>>>
>>>> This does not affect YUYV and similar YUV 4:2:2 formats, which are in
>>>> the expected order.
>>>> ---
>>>>  libavdevice/kmsgrab.c | 16 ++++++++--------
>>>>  1 file changed, 8 insertions(+), 8 deletions(-)
>>>>
>>>> diff --git a/libavdevice/kmsgrab.c b/libavdevice/kmsgrab.c
>>>> index 67a83ef84a..bcb6865f60 100644
>>>> --- a/libavdevice/kmsgrab.c
>>>> +++ b/libavdevice/kmsgrab.c
>>>> @@ -210,14 +210,14 @@ static const struct {
>>>>  #endif
>>>>      { AV_PIX_FMT_RGB24,    DRM_FORMAT_RGB888   },
>>>>      { AV_PIX_FMT_BGR24,    DRM_FORMAT_BGR888   },
>>>> -    { AV_PIX_FMT_0RGB,     DRM_FORMAT_XRGB8888 },
>>>> -    { AV_PIX_FMT_0BGR,     DRM_FORMAT_XBGR8888 },
>>>> -    { AV_PIX_FMT_RGB0,     DRM_FORMAT_RGBX8888 },
>>>> -    { AV_PIX_FMT_BGR0,     DRM_FORMAT_BGRX8888 },
>>>> -    { AV_PIX_FMT_ARGB,     DRM_FORMAT_ARGB8888 },
>>>> -    { AV_PIX_FMT_ABGR,     DRM_FORMAT_ABGR8888 },
>>>> -    { AV_PIX_FMT_RGBA,     DRM_FORMAT_RGBA8888 },
>>>> -    { AV_PIX_FMT_BGRA,     DRM_FORMAT_BGRA8888 },
>>>> +    { AV_PIX_FMT_0RGB,     DRM_FORMAT_BGRX8888 },
>>>> +    { AV_PIX_FMT_0BGR,     DRM_FORMAT_RGBX8888 },
>>>> +    { AV_PIX_FMT_RGB0,     DRM_FORMAT_XBGR8888 },
>>>> +    { AV_PIX_FMT_BGR0,     DRM_FORMAT_XRGB8888 },
>>>> +    { AV_PIX_FMT_ARGB,     DRM_FORMAT_BGRA8888 },
>>>> +    { AV_PIX_FMT_ABGR,     DRM_FORMAT_RGBA8888 },
>>>> +    { AV_PIX_FMT_RGBA,     DRM_FORMAT_ABGR8888 },
>>>> +    { AV_PIX_FMT_BGRA,     DRM_FORMAT_ARGB8888 },
>>>
>>> Is it possible to compile kmsgrab on big-endian hardware?
>>
>> Yes.
> 
> And you assume that on big-endian hardware, above formats
> would still be little-endian?
> 
> Or that they would not be used at all and the big-endian bit would
> be set?
> In that case, it may be simpler to mask the most significant
> bit away in the code and use the endianness-aware formats
> for FFmpeg in the table above (it would reduce the table size
> and make it more readable). Especially as we don't know if
> the bit would also be set for the packed yuv formats.

I think that unlike with RGB565 and similar, the big-endian bit does not need to be set for any of these formats, because the opposite-endian form already exists for each one.  That is, you can always use DRM_FORMAT_XBGR8888 when the order of bytes in memory is RGBX, regardless of the host endianness.

(I think that's a consistent interpretation?)

- Mark