[Libav-user] Problem with h264_videotoolbox and AV_PIX_FMT_YUV420P
steve.green at flosports.tv
Fri Feb 17 23:52:53 EET 2017
This is the basic pipeline that I have:
BGRA frames -> sws_scale() -> YUV frames -> avcodec_encode_video2() -> av_interleaved_write_frame()
.. and this works perfectly with x264 and most of the time with h264_videotoolbox. The issue with h264_videotoolbox happens when I try to stream at 854x480. From looking at the output, it appears that someone along the way is not happy with width not being a multiple of 16. There is a very clear shift/wrapping of pixels.
As an experiment, I changed my use of av_image_fill_arrays() to align on 16 bytes (with a sufficiently larger buffer) and the resulting image is improved but still not correct. The basic shift is gone and the image is legible but the colors are off and I can see an artifact over the image.
Given that x264 works and h264_videotoolbox doesn't, I’m tempted to point fingers at either VTCompressionSession or CVPixelBuffers + YUV, which would explain why I want to try AV_PIX_FMT_VIDEOTOOLBOX. I’ve read pretty much everything I can find on this but I cant find a single mention of 16-byte-multiple widths or strides.
Any idea how to go about debugging this?
More information about the Libav-user