[FFmpeg-devel] [PATCH] avcodec: add new Videotoolbox hwaccel.

[wm4]

On Mon, 27 Jul 2015 18:13:30 +0800
Zhang Rui <bbcallen at gmail.com> wrote:

> 2015-07-15 21:51 GMT+08:00 Clément Bœsch <u at pkh.me>:
> > On Sat, Jun 20, 2015 at 01:33:00PM +0200, Sebastien Zwickert wrote:
> >> Old videtotoolbox patch rebased and updated to target the new HWAccel API.
> >> As VDA is a wrapper of VideoToolbox framework, the changes base vda implementation
> >> upon the videotoolbox implementation to factorize common part of code.
> >>
> >> ---
> >>  Changelog                    |   1 +
> >>  MAINTAINERS                  |   1 +
> >>  Makefile                     |   5 +-
> >>  configure                    |  19 +-
> >>  ffmpeg.h                     |   2 +
> >>  ffmpeg_opt.c                 |   5 +-
> >>  ffmpeg_vda.c                 | 134 --------
> >>  ffmpeg_videotoolbox.c        | 147 +++++++++
> >>  libavcodec/Makefile          |  12 +-
> >>  libavcodec/allcodecs.c       |   5 +
> >>  libavcodec/h263dec.c         |   3 +
> >>  libavcodec/h264_slice.c      |   4 +
> >>  libavcodec/mpeg12dec.c       |   3 +
> >>  libavcodec/vda.c             |   2 +-
> >>  libavcodec/vda_h264.c        | 154 +---------
> >>  libavcodec/vda_internal.h    |  33 --
> >>  libavcodec/vda_vt_internal.h |  57 ++++
> >>  libavcodec/version.h         |   2 +-
> >>  libavcodec/videotoolbox.c    | 705 +++++++++++++++++++++++++++++++++++++++++++
> >>  libavcodec/videotoolbox.h    | 126 ++++++++
> >>  libavutil/pixdesc.c          |   4 +
> >>  libavutil/pixfmt.h           |   2 +
> >
> > I tried on iOS (iPhone 6+ so last one unless I'm mistaken) with a 240fps
> > footage, and it seems the decode calls take more than 5ms, which seems not
> > enough. Do you have any idea what could cause this? (I'm running a timer
> > after/before avcodec_decode_video*())
> 
> 1. kVTDecodeFrame_EnableAsynchronousDecompression can make it faster,
> but not suitable for sync-style API (avcodec_decode_video*())

Makes me wish lavc had an async API. Multithreading is also async to
some degree after all. But it's probably not worth the trouble.

Instead, would it be possible to just use the async VT API, and connect
it to the sync lavc API by doing extra buffering? AFAIK other hw APIs
like mmal and qsv also do this.

> 2. memcpy is murder (in av_image_copy()), but zero-copy needs some
> kind of custom AVFrame.

Isn't this only for the ffmpeg.c wrapper? I wouldn't pay too much
attention for this. Normally, AV_PIX_FMT_VIDEOTOOLBOX_VLD will be used
(which can be directly rendered via OpenGL and such without ever
copying the data to the CPU).

> > [...]
> >
> > --
> > Clément B.
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel