[FFmpeg-user] Lossless encoding to ffv1 changes frames md5

Alberto Mattea alberto at mattea.info
Tue Dec 17 15:18:14 EET 2019 

Hi all, I'm trying to losslessly convert some videos from grassvalley hqx to ffv1.
Unfortunately it seems the conversion is not really lossless, as the frames md5 changes after the conversion.
Here's what I'm doing:

- Calculate md5 of the original video frames

$ ffmpeg -i 01.avi -f framemd5 orig.framemd5
ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 68.100 /  7. 68.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, avi, from '01.avi':
  Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
    Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hqx (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, framemd5, to 'orig.framemd5':
  Metadata:
    encoder         : Lavf58.35.101
    Stream #0:0: Video: rawvideo (Y3[10][16] / 0x100A3359), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 1566720 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.65.100 rawvideo
frame=22610 fps= 50 q=-0.0 Lsize=    1767kB time=00:15:04.40 bitrate=  16.0kbits/s speed=1.98x    
video:172966500kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

- Convert

$ ffmpeg -i 01.avi -c:v ffv1 01ffv1.mkv
ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 68.100 /  7. 68.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, avi, from '01.avi':
  Duration: 00:15:04.40, start: 0.000000, bitrate: 202751 kb/s
    Stream #0:0: Video: hqx (CHQX / 0x58514843), yuv422p16le(10 bpc, progressive), 1632x1200 [SAR 1:1 DAR 34:25], 202756 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hqx (native) -> ffv1 (native))
Press [q] to stop, [?] for help
[ffv1 @ 0x6e1ce80] bits_per_raw_sample > 8, forcing range coder
Output #0, matroska, to '01ffv1.mkv':
  Metadata:
    encoder         : Lavf58.35.101
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p16le(10 bpc), 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.65.100 ffv1
frame=22610 fps= 17 q=-0.0 Lsize=35631195kB time=00:15:04.36 bitrate=322759.1kbits/s speed=0.691x    
video:35630872kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000908%

- Calculate md5 of the converted frames (I've stopped it after a few hundred frames, just to check the beginning)

$ ffmpeg -i 01ffv1.mkv -f framemd5 converted.framemd5       
ffmpeg version N-51066-ge52070e89c-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 68.100 /  7. 68.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, matroska,webm, from '01ffv1.mkv':
  Metadata:
    ENCODER         : Lavf58.35.101
  Duration: 00:15:04.40, start: 0.000000, bitrate: 322745 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv422p10le, 1632x1200, SAR 1:1 DAR 34:25, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      ENCODER         : Lavc58.65.100 ffv1
      DURATION        : 00:15:04.400000000
Stream mapping:
  Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, framemd5, to 'converted.framemd5':
  Metadata:
    encoder         : Lavf58.35.101
    Stream #0:0: Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 1632x1200 [SAR 1:1 DAR 34:25], q=2-31, 979200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      DURATION        : 00:15:04.400000000
      encoder         : Lavc58.65.100 rawvideo
frame=  406 fps= 31 q=-0.0 Lsize=      32kB time=00:00:16.24 bitrate=  16.1kbits/s speed=1.23x    
video:3105900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

- Compare hashes

orig.framemd5:
#format: frame checksums
#version: 2
#hash: MD5
#software: Lavf58.35.101
#tb 0: 1/25
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 1632x1200
#sar 0: 1/1
#stream#, dts,        pts, duration,     size, hash
0,          0,          0,        1,  7833600, 39d64300c8b749b0776423a80746d8dc
0,          1,          1,        1,  7833600, 12e7317d49727740187d2e23439d87d8
0,          2,          2,        1,  7833600, 12e7317d49727740187d2e23439d87d8
0,          3,          3,        1,  7833600, 0b3b0136b008ec9d67e900595be0c923
0,          4,          4,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
0,          5,          5,        1,  7833600, 97cd9b7689d1f49d1e1988a3613ffa72
0,          6,          6,        1,  7833600, 1632e4b3ba7dc28e9dde698f3b7ec9e7
0,          7,          7,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
0,          8,          8,        1,  7833600, 796bda137d768f4ad31eb7166ea16447
0,          9,          9,        1,  7833600, 612c13b4824bd427faa51efa7a09660d
...

converted.framemd5:
#format: frame checksums
#version: 2
#hash: MD5
#software: Lavf58.35.101
#tb 0: 1/25
#media_type 0: video
#codec_id 0: rawvideo
#dimensions 0: 1632x1200
#sar 0: 1/1
#stream#, dts,        pts, duration,     size, hash
0,          0,          0,        1,  7833600, 3ed693fc02ba36cca92a2153afa960ac
0,          1,          1,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
0,          2,          2,        1,  7833600, 23d7482899eb8c854b26444dcdf38db4
0,          3,          3,        1,  7833600, 1fc392823ebc14001ee0e44c68d710e6
0,          4,          4,        1,  7833600, d63335f1c34132680e571549da51eac7
0,          5,          5,        1,  7833600, d63335f1c34132680e571549da51eac7
0,          6,          6,        1,  7833600, 09304bca4578458c353ecc119de0bd87
0,          7,          7,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
0,          8,          8,        1,  7833600, 0ec1e49008d1f70385dc4a1636559a30
0,          9,          9,        1,  7833600, 78b22236d03517e5ceea3bd390ac2d0b
...

Any idea of why this is happening?

More information about the ffmpeg-user mailing list