[FFmpeg-trac] #9573(avcodec:new): libx264 ignores color range flag for gray10 input

[FFmpeg]

#9573: libx264 ignores color range flag for gray10 input
-------------------------------------+-------------------------------------
             Reporter:  Diederick    |                     Type:  defect
  Niehorster                         |
               Status:  new          |                 Priority:  normal
            Component:  avcodec      |                  Version:  git-
             Keywords:  libx264      |  master
  gray10                             |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 When transcoding a file containing gray10 video data in full (pc) color
 range with libx264, the output seems to be in limited (tv) color range
 (i.e. image got a lot darker), despite encoder correctly identifying the
 pc color range. Feel free to add this file to FATE if its an interesting
 test case.

 This is with ffmpeg-2021-12-27-git-617452ce2c-full_build from gyan.dev,
 but has been present for much longer. libx264 version: core 164 r3079
 d9a19f0.

 {{{
 ffmpeg -y -i test.mkv -c:v libx264 -preset veryfast -crf 0 test.mp4
 }}}
 Same thing happens with crf 17, and also when adding -color_range 2 to the
 command.

 relevant part of ffmpeg output (i trimmed it):
 {{{
 Input #0, matroska,webm, from 'test.mkv':
   Metadata:
     ENCODER         : Lavf58.76.100
   Duration: 00:00:00.02, start: 0.000000, bitrate: 1142226 kb/s
   Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), gray10le(pc), 1152x390,
 500 fps, 500 tbr, 1k tbn (default)
     Metadata:
       DURATION        : 00:00:00.020000000
 Stream mapping:
   Stream #0:0 -> #0:0 (ffv1 (native) -> h264 (libx264))
 [libx264 @ 00000237f24afe40] profile High 10, level 5.1, 4:0:0, 10-bit
 [libx264 @ 00000237f24afe40] Output #0, mp4, to 'test.mp4':
   Metadata:
     encoder         : Lavf59.10.100
   Stream #0:0: Video: h264 (avc1 / 0x31637661), gray10le(pc, progressive),
 1152x390, q=2-31, 500 fps, 16k tbn (default)
     Metadata:
       DURATION        : 00:00:00.020000000
       encoder         : Lavc59.15.101 libx264
 }}}
 When probing the resulting file, i get '''(note that pixel format is
 identified as yuv420p10le, not gray10le)''':
 {{{
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
   Metadata:
     major_brand     : isom
     minor_version   : 512
     compatible_brands: isomiso2avc1mp41
     encoder         : Lavf59.10.100
   Duration: 00:00:00.02, start: 0.000000, bitrate: 429001 kb/s
   Stream #0:0[0x1](und): Video: h264 (High 10) (avc1 / 0x31637661),
 yuv420p10le(pc, progressive), 1152x390, 428603 kb/s, 500 fps, 500 tbr, 16k
 tbn (default)
     Metadata:
       handler_name    : VideoHandler
       vendor_id       : [0][0][0][0]
 }}}

 ffmpeg banner:
 {{{
 ffmpeg version 2021-12-27-git-617452ce2c-full_build-www.gyan.dev Copyright
 (c) 2000-2021 the FFmpeg developers
   built with gcc 11.2.0 (Rev2, Built by MSYS2 project)
   configuration: --enable-gpl --enable-version3 --enable-static --disable-
 w32threads --disable-autodetect --enable-fontconfig --enable-iconv
 --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma
 --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt
 --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray
 --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2
 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1
 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2
 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx
 --enable-mediafoundation --enable-libass --enable-frei0r --enable-
 libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf
 --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-
 ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2
 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo
 --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug
 --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame
 --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-
 amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb
 --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa
 --enable-libbs2b --enable-libflite --enable-libmysofa --enable-
 librubberband --enable-libsoxr --enable-chromaprint
   libavutil      57. 13.100 / 57. 13.100
   libavcodec     59. 15.101 / 59. 15.101
   libavformat    59. 10.100 / 59. 10.100
   libavdevice    59.  1.100 / 59.  1.100
   libavfilter     8. 21.100 /  8. 21.100
   libswscale      6.  1.102 /  6.  1.102
   libswresample   4.  0.100 /  4.  0.100
   libpostproc    56.  0.100 / 56.  0.100
 }}}
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9573>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker