[FFmpeg-trac] #3345(swscale:open): Bias in planar YUV to YUV bit depth conversion

[FFmpeg]

#3345: Bias in planar YUV to YUV bit depth conversion
------------------------------------+-----------------------------------
             Reporter:  abc123      |                    Owner:  (none)
                 Type:  defect      |                   Status:  open
             Priority:  normal      |                Component:  swscale
              Version:  git-master  |               Resolution:
             Keywords:  bounty      |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |
------------------------------------+-----------------------------------
Comment (by pdr0):

 Replying to [comment:11 Balling]:
 > It is not OP but Kieran that claimes this libav is not affected.
 >

 Actually it's not Kieran, but Issac Dian who makes that claim ( who I
 thought was the OP, abc123). It's Kieran's reply that explains why. But
 swscale still appears to have the issue, and it's separate issue from
 dither

 > There is no libav anymore or at least no real development there. Most of
 patches from libav are in ffmpeg anyways.
 >

 True, but that claim about libav was made 8 years ago, when libav
 development was more active

 If you test a ffmpeg binary around that time from the ffmpeg build in the
 OP (Jan 14 2014) - the results are different than ffmpeg today, either
 default or dither disabled

 The issue is still present with libav 2018, so maybe there was more than
 one issue. Maybe observations were being confused with dithering, or maybe
 there were commits that broke and fixed things over time. My guess is
 problem was always there



 The ticket is for YUV to YUV - so ideally that should be examined in YUV.
 There shouldn't be any talk about RGB conversions or players or PNG's, or
 you just obfuscate the issue and introduce other variables

 The sample provided is 4:2:0, so the U,V planes at 640x360 have to be
 examined and compared to a "divide by 4" result on each plane.

 The Y plane is a vertical grad goes from Y=525  at Ypos 0 , to Y=504 at
 Ypos 719
 U plane goes from U=501 at Ypos 0 , to U=513 at Ypos 359
 V planes goes from V=560 at Ypos 0 , to V=508 at Ypos 359


 The claim was a bias of -0.375 compared to the "divide by 4" result.

 If that is still a bug, let's pick some values that should demonstrate the
 error .

 523/4 = 130.75 => should round up to 131 . -0.375 Bias would mean 130.375
 and rounding down to 130

 503/4 = 125.75 => should round up to 126 . -0.375 Bias would mean 125.375
 and rounding down to 125

 559/4 = 139.75 => should round up to 140 . -0.375 Bias would mean 139.375
 and rounding down to 139

 Y=523 occurs between Ypos 117 to 149 (1280x720)
 U=503 occurs between Ypos 74 to 100 (640x360)
 V=559 occurs between Ypos 20 to 30 (640x360)

 ffmpeg -vf scale, nodither
 Y=130
 U=125
 V=139

 ffmpeg zscale, nodither
 Y=131
 U=126
 V=140

 I checked a few more coordinates and it looks like issue still present
 with swscale, but absent with zscale .
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/3345#comment:12>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker