[FFmpeg-devel] [PATCH] libavutil/hwcontext_qsv: fix a bug for mapping qsv frame to vaapi

[Chen, Wenbin]

> On Fri, 2021-09-10 at 02:19 +0000, Chen, Wenbin wrote:
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > James Almer
> > > Sent: Thursday, September 9, 2021 8:48 PM
> > > To: ffmpeg-devel at ffmpeg.org
> > > Subject: Re: [FFmpeg-devel] [PATCH] libavutil/hwcontext_qsv: fix a bug
> for
> > > mapping qsv frame to vaapi
> > >
> > > On 9/9/2021 6:13 AM, Wenbin Chen wrote:
> > > > Command below failed.
> > > > ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128
> > > > -init_hw_device qsv=qs at va -hwaccel qsv -hwaccel_device qs
> > > > -filter_hw_device va -c:v h264_qsv
> > > > -i 1080P.264 -vf "hwmap,format=vaapi" -c:v h264_vaapi output.264
> > > >
> > > > Cause: Assign pair->first directly to data[3] in vaapi frame.
> > > > pair->first is *VASurfaceID while data[3] in vaapi frame is
> > > > VASurfaceID. I fix this line of code. Now the command above works.
> > > >
> > > > Signed-off-by: Wenbin Chen <wenbin.chen at intel.com>
> > > > ---
> > > >   libavutil/hwcontext_qsv.c | 2 +-
> > > >   1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> > > > index d431e71eab..6539cae619 100644
> > > > --- a/libavutil/hwcontext_qsv.c
> > > > +++ b/libavutil/hwcontext_qsv.c
> > > > @@ -781,7 +781,7 @@ static int qsv_map_from(AVHWFramesContext
> *ctx,
> > > >       case AV_HWDEVICE_TYPE_VAAPI:
> > > >       {
> > > >           mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId;
> > > > -        child_data = pair->first;
> > > > +        child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)pair->first;
> > >
> > > You can probably remove the intptr_t casting.
> >
> > If intptr_t is removed, it will report compile warning
> > "cast to pointer from integer of different size"
> 
> How about to add a comment here? If so, others can understand why this
> casting
> is needed.
> 
> Thanks
> Haihao

Ok, I will send path V2

> 
> 
> >
> > >
> > > Also, shouldn't this same fix be done for all three child device types
> > > used in this function? Which for that matter, child_data seems to be set
> > > for a d3d11va child device, but then never used.
> >
> > Dxva and d3d11 frames store the pointer to surface while vaapi frames
> store
> > surface,
> > so this part of code for dxva and d3d11va should be correct.
> >
> > D3d11 and dxva share dxva codec, the child_data is used in this part of
> code.
> > >
> > > >           break;
> > > >       }
> > > >   #endif
> > > >
> > >
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel at ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".