[FFmpeg-devel] [PATCHv6 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

Alexis Ballier aballier at gentoo.org
Mon Aug 28 12:26:41 EEST 2017


Hi,


On Sun, 27 Aug 2017 21:41:06 +0200
Jorge Ramirez <jorge.ramirez-ortiz at linaro.org> wrote:

> On 08/27/2017 09:09 PM, Jorge Ramirez wrote:
> > On 08/25/2017 05:35 PM, wm4 wrote:  
> >>> +static int buffer_ops_v4l2buf_to_avframe(AVFrame *frame,
> >>> V4L2Buffer *avbuf) +{
> >>> +    int i, ret;
> >>> +
> >>> +    av_frame_unref(frame);
> >>> +
> >>> +    /* 1. get references to the actual data */
> >>> +    for (i = 0; i < avbuf->num_planes; i++) {
> >>> +        ret = avbuf->ops.buf_to_bufref(avbuf, i, &frame->buf[i]);
> >>> +        if (ret)
> >>> +            return ret;
> >>> +
> >>> +        frame->linesize[i] = avbuf->bytesperline[i];
> >>> +        frame->data[i] = frame->buf[i]->data;
> >>> +    }
> >>> +
> >>> +    /* 1.1 fixup special cases */
> >>> +    switch (avbuf->context->av_pix_fmt) {
> >>> +    case AV_PIX_FMT_NV12:
> >>> +        if (avbuf->num_planes > 1)
> >>> +            break;
> >>> +        frame->linesize[1] = avbuf->bytesperline[0];
> >>> +        frame->data[1] = frame->buf[0]->data +
> >>> avbuf->bytesperline[0] * avbuf->context->format.fmt.pix_mp.height;
> >>> +        break;
> >>> +    default:
> >>> +        break;
> >>> +    }
> >>> +
> >>> +    /* 2. get frame information */
> >>> +    frame->key_frame = !!(avbuf->buf.flags &
> >>> V4L2_BUF_FLAG_KEYFRAME);
> >>> +    frame->format = avbuf->context->av_pix_fmt;
> >>> +
> >>> +    /* these values are updated also during re-init in
> >>> process_video_event */
> >>> +    frame->height = avbuf->context->height;
> >>> +    frame->width = avbuf->context->width;
> >>> +    frame->pts = get_pts(avbuf);
> >>> +
> >>> +    /* 3. report errors upstream */
> >>> +    if (avbuf->buf.flags & V4L2_BUF_FLAG_ERROR) {
> >>> +        av_log(avbuf->context->log_ctx, AV_LOG_ERROR, "%s:
> >>> driver decode error\n", avbuf->context->name);
> >>> +        frame->decode_error_flags |=
> >>> FF_DECODE_ERROR_INVALID_BITSTREAM;
> >>> +    }
> >>> +
> >>> +    return 0;
> >>> +}  
> >> This function seems to lack setting typically required metadata
> >> like colorspace.
> >>  
> >
> > ok I will retrieve the colorspace from the v4l2 format structure
> > and set it in the frame.  
> 
> um, I dont see a 1:1 mapping between the colorspaces reported from
> v4l2 and what ffmpeg expects (I am also not a subject matter expert
> on this :( ).
> 
> Could I get some guidance on the translation table below please?
> btw in my simple tests - ffplay decoded vp8, vp9, mpeg4, mpeg2, h263, 
> h264 and hevc I didnt need this field so I am not sure if I am
> getting it right.


I don't think it will affect decoding, but the image will be something like too pale if it's set wrongly


> static inline enum AVColorSpace get_colorspace(V4L2Buffer *buf)
> {
>      enum v4l2_colorspace cs;
> 
>      cs = V4L2_TYPE_IS_MULTIPLANAR(buf->context->type) ?
>          buf->context->format.fmt.pix_mp.colorspace :
>          buf->context->format.fmt.pix.colorspace;
> 
>      /* */
>      switch (cs) {
>      case V4L2_COLORSPACE_SMPTE170M: return AVCOL_SPC_SMPTE170M;
>      case V4L2_COLORSPACE_SMPTE240M: return AVCOL_SPC_SMPTE240M;
>      case V4L2_COLORSPACE_470_SYSTEM_BG: return AVCOL_SPC_BT470BG;
>      case V4L2_COLORSPACE_BT2020: return AVCOL_SPC_BT2020_CL;
>      case V4L2_COLORSPACE_REC709: return AVCOL_SPC_BT709;
>      case V4L2_COLORSPACE_ADOBERGB:
>      case V4L2_COLORSPACE_SRGB:
>      case V4L2_COLORSPACE_DCI_P3:
>      case V4L2_COLORSPACE_JPEG:
>      case V4L2_COLORSPACE_RAW:
>      default:
>          return AVCOL_SPC_RGB;
>      }


FWIW, here is the conversion I had been using in other projects:
                                                                                                                         
    switch(f->colorspace)                                                                                                
    {                                                                                                                    
        case V4L2_COLORSPACE_SRGB:                                                                                       
            return AVCOL_SPC_RGB;                                                                                        
        case V4L2_COLORSPACE_REC709:                                                                                     
            return AVCOL_SPC_BT709;                                                                                      
        case V4L2_COLORSPACE_470_SYSTEM_M:                                                                               
            return AVCOL_SPC_FCC;                                                                                        
        case V4L2_COLORSPACE_470_SYSTEM_BG:                                                                              
            return AVCOL_SPC_BT470BG;                                                                                    
        case V4L2_COLORSPACE_SMPTE170M:                                                                                  
            return AVCOL_SPC_SMPTE170M;                                                                                  
        case V4L2_COLORSPACE_SMPTE240M:                                                                                  
            return AVCOL_SPC_SMPTE240M;                                                                                 
        case V4L2_COLORSPACE_BT2020:                                                                                     
            if(f->ycbcr_enc == V4L2_YCBCR_ENC_BT2020_CONST_LUM)                                                          
                return AVCOL_SPC_BT2020_CL;                                                                              
            return AVCOL_SPC_BT2020_NCL;                                                                                 
        default: return AVCOL_SPC_UNSPECIFIED;
   }                                                                                                                    
                                                                      

you'd likely need AVCOL_PRI*:

    switch(f->ycbcr_enc)                                                                                                 
    {                                                                                                                    
        case V4L2_YCBCR_ENC_XV709:                                                                                       
        case V4L2_YCBCR_ENC_709  :                                                                                       
            return AVCOL_PRI_BT709;                                                                                      
        case V4L2_YCBCR_ENC_XV601:                                                                                       
        case V4L2_YCBCR_ENC_601:                                                                                         
            return AVCOL_PRI_BT470M;                                                                                     
        default: break;                                                                                                  
    }                                                                                                                    
                                                                                                                         
    switch(f->colorspace)                                                                                                
    {                                                                                                                    
        case V4L2_COLORSPACE_470_SYSTEM_BG:                                                                              
            return AVCOL_PRI_BT470BG;                                                                                    
        case V4L2_COLORSPACE_SMPTE170M:                                                                                  
            return AVCOL_PRI_SMPTE170M;                                                                                  
        case V4L2_COLORSPACE_SMPTE240M:                                                                                  
            return AVCOL_PRI_SMPTE240M;                                                                                  
        case V4L2_COLORSPACE_BT2020:                                                                                     
            return AVCOL_PRI_BT2020;                                                                                     
        default: break;                                                                                                  
    }                                                                                                                    
    return AVCOL_PRI_UNSPECIFIED;


AVCOL_TRC*:

    switch(f->xfer_func)                                                                                                 
    {                                                                                                                    
        case V4L2_XFER_FUNC_709:                                                                                         
            switch(f->bit_depth)
            {                                                                                                            
                case 10: return AVCOL_TRC_BT2020_10;                                                                     
                case 12: return AVCOL_TRC_BT2020_12;                                                                     
                default: break;                                                                                          
            }                                                                                                            
            return AVCOL_TRC_BT709;                                                                                      
        case V4L2_XFER_FUNC_SRGB:                                                                                        
            return AVCOL_TRC_IEC61966_2_1;                                                                               
        default: break;                                                                                                  
    }                                                                                                                    
                                                                                                                         
    switch(f->colorspace)                                                                                                
    {                                                                                                                    
        case V4L2_COLORSPACE_470_SYSTEM_M:                                                                               
            return AVCOL_TRC_GAMMA22;                                                                                    
        case V4L2_COLORSPACE_470_SYSTEM_BG:                                                                              
            return AVCOL_TRC_GAMMA28;                                                                                    
        case V4L2_COLORSPACE_SMPTE170M:                                                                                  
            return AVCOL_TRC_SMPTE170M;                                                                                  
        case V4L2_COLORSPACE_SMPTE240M:                                                                                  
            return AVCOL_TRC_SMPTE240M;                                                                                  
        default: break;                                                                                                  
    }                                                                                                                    
                                                                                                                         
    switch(f->ycbcr_enc)                                                                                                 
    {                                                                                                                    
        case V4L2_YCBCR_ENC_XV709:                                                                                       
        case V4L2_YCBCR_ENC_XV601:                                                                                       
            return AVCOL_TRC_BT1361_ECG;                                                                                 
        default: break;                                                                                                  
    }                                                                                                                    
    return AVCOL_TRC_UNSPECIFIED;

AVCOL_RANGE*:

    switch(f->quantization)                                                                                              
    {                                                                                                                    
        case V4L2_QUANTIZATION_LIM_RANGE:                                                                                
            return AVCOL_RANGE_MPEG;                                                                                     
        case V4L2_QUANTIZATION_FULL_RANGE:                                                                               
            return AVCOL_RANGE_JPEG;                                                                                     
        default: break;                                                                                                  
    }                                                                                                                    
                                                                                                                         
    return AVCOL_RANGE_UNSPECIFIED;                      



Feel free to re-use, point and/or fix bugs here (it's been long since I wrote this and I don't think I've ever visually checked all the possibilities).
This might also be useful for the V4L2 output and capture drivers in lavd.

Bests,

Alexis.


More information about the ffmpeg-devel mailing list