[FFmpeg-devel] [PATCH] Add android_capture indev

Felix Matouschek felix at matouschek.org
Tue Dec 19 22:04:06 EET 2017


Hello Michael,

could you take a look at the patch?

Felix

> Am 30.11.2017 um 18:15 schrieb Michael Niedermayer <michael at niedermayer.cc>:
> 
> On Thu, Nov 30, 2017 at 10:15:48AM +0100, Felix Matouschek wrote:
>> Sorry, my mail client swallowed the attachment, sent it again.
>> 
> 
>> Changelog                    |    1 
>> MAINTAINERS                  |    1 
>> configure                    |    6 
>> doc/indevs.texi              |   40 ++
>> libavdevice/Makefile         |    1 
>> libavdevice/alldevices.c     |    1 
>> libavdevice/android_camera.c |  816 +++++++++++++++++++++++++++++++++++++++++++
>> libavdevice/version.h        |    2 
>> 8 files changed, 867 insertions(+), 1 deletion(-)
>> 2fea6cb3990aed83c3a3c492aa482f619a885ed7  0001-avdevice-add-android_camera-indev.patch
>> From b70da28e33e07b4565daffc94c8ffe3c8df747ff Mon Sep 17 00:00:00 2001
>> From: Felix Matouschek <felix at matouschek.org>
>> Date: Thu, 30 Nov 2017 10:03:54 +0100
>> Subject: [PATCH] avdevice: add android_camera indev
>> To: ffmpeg-devel at ffmpeg.org
>> 
>> This commit adds an indev for Android devices on API level 24+ which
>> uses the Android NDK Camera2 API to capture video from builtin cameras
>> 
>> Signed-off-by: Felix Matouschek <felix at matouschek.org>
>> ---
>> Changelog                    |   1 +
>> MAINTAINERS                  |   1 +
>> configure                    |   6 +
>> doc/indevs.texi              |  40 +++
>> libavdevice/Makefile         |   1 +
>> libavdevice/alldevices.c     |   1 +
>> libavdevice/android_camera.c | 816 +++++++++++++++++++++++++++++++++++++++++++
>> libavdevice/version.h        |   2 +-
>> 8 files changed, 867 insertions(+), 1 deletion(-)
>> create mode 100644 libavdevice/android_camera.c
>> 
>> diff --git a/Changelog b/Changelog
>> index 6592d868da..f58cd810e0 100644
>> --- a/Changelog
>> +++ b/Changelog
>> @@ -6,6 +6,7 @@ version <next>:
>> - Dropped support for OpenJPEG versions 2.0 and below. Using OpenJPEG now
>>   requires 2.1 (or later) and pkg-config.
>> - VDA dropped (use VideoToolbox instead)
>> +- Add android_camera indev
>> 
>> 
>> version 3.4:
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 1d2ff78b0e..d6cb135964 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -281,6 +281,7 @@ libavdevice
>> 
>> 
>>   avfoundation.m                        Thilo Borgmann
>> +  android_camera.c                      Felix Matouschek
>>   decklink*                             Marton Balint
>>   dshow.c                               Roger Pack (CC rogerdpack at gmail.com)
>>   fbdev_enc.c                           Lukasz Marek
>> diff --git a/configure b/configure
>> index 7a53bc76c7..d52b18fab3 100755
>> --- a/configure
>> +++ b/configure
>> @@ -3068,6 +3068,8 @@ xmv_demuxer_select="riffdec"
>> xwma_demuxer_select="riffdec"
>> 
>> # indevs / outdevs
>> +android_camera_indev_deps="android camera2ndk mediandk pthreads"
>> +android_camera_indev_extralibs="-landroid -lcamera2ndk -lmediandk"
>> alsa_indev_deps="alsa"
>> alsa_outdev_deps="alsa"
>> avfoundation_indev_deps="avfoundation corevideo coremedia pthreads"
>> @@ -5836,6 +5838,10 @@ check_lib shell32  "windows.h shellapi.h" CommandLineToArgvW   -lshell32
>> check_lib wincrypt "windows.h wincrypt.h" CryptGenRandom       -ladvapi32
>> check_lib psapi    "windows.h psapi.h"    GetProcessMemoryInfo -lpsapi
>> 
>> +check_lib android android/native_window.h ANativeWindow_acquire -landroid
>> +check_lib mediandk "stdint.h media/NdkImage.h" AImage_delete -lmediandk
>> +check_lib camera2ndk "stdbool.h stdint.h camera/NdkCameraManager.h" ACameraManager_create -lcamera2ndk
>> +
>> enabled appkit       && check_apple_framework AppKit
>> enabled audiotoolbox && check_apple_framework AudioToolbox
>> enabled avfoundation && check_apple_framework AVFoundation
>> diff --git a/doc/indevs.texi b/doc/indevs.texi
>> index d308bbf7de..07056d762e 100644
>> --- a/doc/indevs.texi
>> +++ b/doc/indevs.texi
>> @@ -63,6 +63,46 @@ Set the number of channels. Default is 2.
>> 
>> @end table
>> 
>> + at section android_camera
>> +
>> +Android camera input device.
>> +
>> +This input devices uses the Android Camera2 NDK API which is
>> +available on devices with API level 24+. The availability of
>> +android_camera is autodetected during configuration.
>> +
>> +This device allows capturing from all cameras on an Android device,
>> +which are integrated into the Camera2 NDK API.
>> +
>> +The available cameras are enumerated internally and can be selected
>> +with the @var{camera_index} parameter. The input file string is
>> +discarded.
>> +
>> +Generally the back facing camera has index 0 while the front facing
>> +camera has index 1.
>> +
>> + at subsection Options
>> +
>> + at table @option
>> +
>> + at item video_size
>> +Set the video size given as a string such as 640x480 or hd720.
>> +Falls back to the first available configuration reported by
>> +Android if requested video size is not available or by default.
>> +
>> + at item framerate
>> +Set the video framerate.
>> +Falls back to the first available configuration reported by
>> +Android if requested framerate is not available or by default (-1).
>> +
>> + at item camera_index
>> +Set the index of the camera to use. Default is 0.
>> +
>> + at item input_queue_size
>> +Set the maximum number of frames to buffer. Default is 5.
>> +
>> + at end table
>> +
>> @section avfoundation
>> 
>> AVFoundation input device.
>> diff --git a/libavdevice/Makefile b/libavdevice/Makefile
>> index 8228d62147..f11a6f2a86 100644
>> --- a/libavdevice/Makefile
>> +++ b/libavdevice/Makefile
>> @@ -14,6 +14,7 @@ OBJS-$(CONFIG_SHARED)                    += reverse.o
>> # input/output devices
>> OBJS-$(CONFIG_ALSA_INDEV)                += alsa_dec.o alsa.o timefilter.o
>> OBJS-$(CONFIG_ALSA_OUTDEV)               += alsa_enc.o alsa.o
>> +OBJS-$(CONFIG_ANDROID_CAMERA_INDEV)      += android_camera.o
>> OBJS-$(CONFIG_AVFOUNDATION_INDEV)        += avfoundation.o
>> OBJS-$(CONFIG_BKTR_INDEV)                += bktr.o
>> OBJS-$(CONFIG_CACA_OUTDEV)               += caca.o
>> diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c
>> index b767b6a718..2c8d9035da 100644
>> --- a/libavdevice/alldevices.c
>> +++ b/libavdevice/alldevices.c
>> @@ -42,6 +42,7 @@ static void register_all(void)
>> {
>>     /* devices */
>>     REGISTER_INOUTDEV(ALSA,             alsa);
>> +    REGISTER_INDEV   (ANDROID_CAMERA,   android_camera);
>>     REGISTER_INDEV   (AVFOUNDATION,     avfoundation);
>>     REGISTER_INDEV   (BKTR,             bktr);
>>     REGISTER_OUTDEV  (CACA,             caca);
>> diff --git a/libavdevice/android_camera.c b/libavdevice/android_camera.c
>> new file mode 100644
>> index 0000000000..eaef032a4c
>> --- /dev/null
>> +++ b/libavdevice/android_camera.c
>> @@ -0,0 +1,816 @@
>> +/*
>> + * Android camera input device
>> + *
>> + * Copyright (C) 2017 Felix Matouschek
>> + *
>> + * This file is part of FFmpeg.
>> + *
>> + * FFmpeg is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2.1 of the License, or (at your option) any later version.
>> + *
>> + * FFmpeg is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with FFmpeg; if not, write to the Free Software
>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
>> + */
>> +
>> +#include <errno.h>
>> +#include <pthread.h>
>> +#include <stdatomic.h>
>> +#include <stdbool.h>
>> +#include <stdint.h>
>> +
>> +#include <camera/NdkCameraDevice.h>
>> +#include <camera/NdkCameraManager.h>
>> +#include <media/NdkImage.h>
>> +#include <media/NdkImageReader.h>
>> +
>> +#include "libavformat/avformat.h"
>> +#include "libavformat/internal.h"
>> +#include "libavutil/avstring.h"
>> +#include "libavutil/display.h"
>> +#include "libavutil/imgutils.h"
>> +#include "libavutil/log.h"
>> +#include "libavutil/opt.h"
>> +#include "libavutil/parseutils.h"
>> +#include "libavutil/threadmessage.h"
>> +#include "libavutil/time.h"
>> +
>> +#include "version.h"
>> +
>> +/* This image format is available on all Android devices
>> + * supporting the Camera2 API */
>> +#define IMAGE_FORMAT_ANDROID AIMAGE_FORMAT_YUV_420_888
>> +#define IMAGE_FORMAT_FFMPEG AV_PIX_FMT_YUV420P
>> +#define IMAGE_NUM_PLANES 3
>> +
>> +#define MAX_BUF_COUNT 2
>> +#define VIDEO_STREAM_INDEX 0
>> +#define VIDEO_TIMEBASE 1000000000
>> +
>> +typedef struct AndroidCameraCtx {
>> +    const AVClass *class;
>> +
>> +    int requested_width;
>> +    int requested_height;
>> +    AVRational framerate;
>> +    int camera_index;
>> +    int input_queue_size;
>> +
>> +    uint8_t lens_facing;
>> +    int32_t sensor_orientation;
>> +    int width;
>> +    int height;
>> +    int32_t framerate_range[2];
>> +
>> +    ACameraManager *camera_mgr;
>> +    char *camera_id;
>> +    ACameraMetadata *camera_metadata;
>> +    ACameraDevice *camera_dev;
>> +    ACameraDevice_StateCallbacks camera_state_callbacks;
>> +    AImageReader *image_reader;
>> +    AImageReader_ImageListener image_listener;
>> +    ANativeWindow *image_reader_window;
>> +    ACaptureSessionOutputContainer *capture_session_output_container;
>> +    ACaptureSessionOutput *capture_session_output;
>> +    ACameraOutputTarget *camera_output_target;
>> +    ACaptureRequest *capture_request;
>> +    ACameraCaptureSession_stateCallbacks capture_session_state_callbacks;
>> +    ACameraCaptureSession *capture_session;
>> +
>> +    AVThreadMessageQueue *input_queue;
>> +    atomic_int exit;
>> +    int display_matrix_sent;
>> +} AndroidCameraCtx;
>> +
>> +static const char *camera_status_string(camera_status_t val)
>> +{
>> +    switch(val) {
>> +        case ACAMERA_OK:
>> +            return AV_STRINGIFY(ACAMERA_OK);
>> +        case ACAMERA_ERROR_UNKNOWN:
>> +            return AV_STRINGIFY(ACAMERA_ERROR_UNKNOWN);
>> +        case ACAMERA_ERROR_INVALID_PARAMETER:
>> +            return AV_STRINGIFY(ACAMERA_ERROR_INVALID_PARAMETER);
> 
> what i meant was something like
> 
> #define RETURN_CASE(x) case x: return AV_STRINGIFY(x);
> switch(val) {
>    RETURN_CASE(ACAMERA_OK)
>    RETURN_CASE(ACAMERA_ERROR_UNKNOWN)
>    RETURN_CASE(ACAMERA_ERROR_INVALID_PARAMETER)
> 
> 
> [...]
> 
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> If you fake or manipulate statistics in a paper in physics you will never
> get a job again.
> If you fake or manipulate statistics in a paper in medicin you will get
> a job for life at the pharma industry.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel



More information about the ffmpeg-devel mailing list