[FFmpeg-trac] #3794(swscale:new): SWSCALE: Quantization Errors in Y Studio Range

[FFmpeg]

#3794: SWSCALE: Quantization Errors in Y Studio Range
------------------------------------+-----------------------------------
             Reporter:  troy_s      |                    Owner:
                 Type:  defect      |                   Status:  new
             Priority:  normal      |                Component:  swscale
              Version:  git-master  |               Resolution:
             Keywords:  range       |               Blocked By:
             Blocking:              |  Reproduced by developer:  0
Analyzed by developer:  0           |
------------------------------------+-----------------------------------

Comment (by troy_s):

 Attaching the matrix used for conversion.


 {{{
 0.1825858824    0.6142305882    0.0620070588    +0.062745098
 -0.1006437324   -0.3385719539   0.4392156863    +0.5019607843
 0.4392156863    -0.3989421626   -0.0402735237   +0.5019607843
 }}}

 The above is the float version of:

 {{{
 0.2126*(219/255)              0.7152*(219/255)
 0.0722*(219/255)           +(16/255)
 -(0.2126/1.8556)*(224/255)    -(0.7152/1.8556)*(224/255)    0.5*(224/255)
 +(128/255)
 0.5*(224/255)                 -(0.7152/1.5748)*(224/255)
 -(0.0722/1.5748)*(224/255) +(128/255)
 }}}

 (For the curious, the 1.8556 and 1.5748 is nothing more than (Coeff.Red +
 Coeff.Green) * 2 and (Coeff.Blue + Coeff.Green) * 2 to normalize the
 values around the origin.)

--
Ticket URL: <https://trac.ffmpeg.org/ticket/3794#comment:13>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker