[FFmpeg-devel] HEVC decoder for Raspberry Pi

[jc at kynesim.co.uk]

From: Philip Langdale
Sent: 15 November 2018 22:42
To: FFmpeg development discussions and patches
Subject: Re: [FFmpeg-devel] HEVC decoder for Raspberry Pi

On 2018-11-15 05:48, John Cox wrote:
>> 
>> Now one way it could be integrated would be as a seperate decoder
> That is how I've currently built it and therefore probably the easiest
> option.
> 
>> another is inside the hevc decoder
> It started life there but became a very uneasy fit with too many 
> ifdefs.
>> a 3rd is, similar to the hwaccel stuff
>> and a 4th would be that the decoder could be an external lib that
>> is used through hwaccel similar to other hwaccel libs
> Possibly - this is where I wanted advice as my attempts to understand
> how that lot is meant to work simply ended in confusion or a feeling
> that what I wanted to do was a very bad fit with the current framework 
> -
> some of the issue with that is in vps/sps/pps setup where I build
> somewhat different tables to the common code that is used by most other
> h/w decodes.
> 
>> you need to obtain the communities preferrance here not just my
>> oppinion ...
>> especially comments from people activly working on hwaccel stuff
>> are needed here
> I welcome their comments
> 

Hi John,

I'm not exactly an hwaccel expert (Hopefully jkqxz will chime in), but
your scenario looks like one that fits naturally into the hwaccel model
on the surface. You don't want to duplicate the whole decoder and you
want to accelerate specific steps using an offload mechanism and there's
a custom pix fmt involved. It's not a traditional GPU offload but it's
a close enough analogue that it's where I'd start.

So, I think you are basically doing two things:

1) Defining a new hwcontext that encapsulates your offload
    co-processor and an accompanying hw pix fmt for the sand format
    in GPU memory. The hwcontext API gives you hooks to define
    transfers to/from system memory. This replaces the filters you
    have in your current implementation.

2) You then define an an hwaccel that uses the new hwcontext and
    format. You mentioned you needed to organise the hevc metadata
    differently from how ffmpeg does, but this is true for all
    hwaccels - if you look at the existing ones, there's lots of
    boilerplate code doing various transformations and organisations
    on this; that's just life. It's tedious but you only do it once.

    You then need to write your actual decode. Unlike a full hwaccel,
    it sounds like parts of the process are still CPU code, and maybe
    could be the same code in the regular software decoding path. You
    might be able to refactor to use it, or have to duplicate it, but
    that's not the end of the world.

Hopefully, that's of some use to you.

--phil
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

First off, sorry if this isn't formatted right, I'm on my phone so I'll give you a proper reply come Monday.

Thanks for the info. One of the main issues here is that the Pi is critically short of processing power to do this trick usefully so I do care (a lot) about every single percent of performance. Almost all conversations or switching in/out of common code will cost me.

Little of the original decoder survives completely intact, dpb stuff is close to identical, most of the sps etc. parse is the same, the overall structure of the main decode is similar but nearly all the detail has been hacked.

Regards

John Cox