[FFmpeg-devel] [RFC 2/4] hwaccel: try get_buffer*() if hwaccel alloc_frame() yields ENOSYS.

[Gwenole Beauchesne]

By default, lavc can manage frame buffer allocations itself without
explicit intervention from the user and custom get_buffer*() hooks.
In view to simplifying usages, there is a desire to operate on the
same model for hardware accelerated pipelines, i.e. delegate all HW
resources management to lavc.

AVHWAccel.alloc_frame() can serve this purpose just fine. However,
if the previous model is needed, whereby get_buffer*() hooks are
required, there is zero chance to call into them if alloc_frame() is
defined.

One way to solve this problem is simply to detect this usage model
from the hwaccel implementation and return AVERROR(ENOSYS) in the
.alloc_frame() hook so that to notify get_buffer_internal() to try
to call into .get_buffer*() hooks.

This is an internal semantical change only, and does not affect any
existing hwaccel implementation.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne at intel.com>
---
 libavcodec/utils.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index d6be93d..321bdde 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -903,7 +903,9 @@ static int get_buffer_internal(AVCodecContext *avctx, AVFrame *frame, int flags)
     if (hwaccel) {
         if (hwaccel->alloc_frame) {
             ret = hwaccel->alloc_frame(avctx, frame);
-            goto end;
+            if (ret != AVERROR(ENOSYS))
+                goto end;
+            /* Try out through get_buffer2() interfaces */
         }
     } else
         avctx->sw_pix_fmt = avctx->pix_fmt;
-- 
1.9.1