[FFmpeg-devel] [PATCH] fate/jpeg2000dec: add missing ISO/IEC 15444-4 conformance tests

Pierre-Anthony Lemieux pal at sandflow.com
Mon Nov 18 07:57:52 EET 2024 

On Mon, Nov 11, 2024 at 12:19 AM WATANABE Osamu
<owatanab at es.takushoku-u.ac.jp> wrote:
>
> I have confirmed that these failures in FATE were due to the insufficient floating point precision of a 32-bit environment.
>
> The commit 82467b635efced67c1767cb810af1f3c31a2e493 introduces the improved dequantization in FF_DWT97_INT path and makes FFmpeg compliant for ISO/IEC 15444-4 (Conformance testing.)
>
> Actually, both 64 and 32-bit FFmpeg with the commit pass the conformance testing.
>
> Current FATE tests for JPEG 2000 are to check the CRC of reconstructed images.
> Checking CRC is unsuitable for lossy codestream because even a patch that can improve the quality of a reconstructed image cannot pass the FATE. Passing FATE is, of course, important, but passing the official (ISO's) conformance tests is essential, too.
>
> I recommend introducing a way to allow some tolerance in FATE tests for JPEG 2000 lossy codestreams.
> For example, including the original files and using the PSNR value in FATE will be an option.

The original files are approximately 10 MB. Any objection/concerns to
adding them to FATE, which can then apply the conformance requirements
specified in the standard?

>
>
> > On Nov 10, 2024, at 10:40, Michael Niedermayer <michael at niedermayer.cc> wrote:
> >
> > On Fri, Nov 08, 2024 at 01:09:34PM -0800, pal at sandflow.com wrote:
> >> From: Pierre-Anthony Lemieux <pal at palemieux.com>
> >>
> >> ---
> >> tests/fate/jpeg2000.mak                  | 122 ++++++++++++++++++++++-
> >> tests/ref/fate/jpeg2000dec-ds0_hm_15_b8  |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_02_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_02_b12 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_03_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_03_b14 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_04_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_04_b12 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_05_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_05_b12 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_07_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_07_b15 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_07_b16 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_08_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_08_b15 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_08_b16 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_09_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_10_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_11_b10 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_12_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_14_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_15_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_15_b14 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds0_ht_16_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_01_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_01_b12 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_02_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_02_b12 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_03_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_03_b12 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_04_b9  |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_05_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-ds1_ht_06_b11 |   6 ++
> >> tests/ref/fate/jpeg2000dec-hifi_ht1_02   |   6 ++
> >> tests/ref/fate/jpeg2000dec-hifi_p1_02    |   6 ++
> >> tests/ref/fate/jpeg2000dec-p1_01         |   6 ++
> >> tests/ref/fate/jpeg2000dec-p1_02         |   6 ++
> >> tests/ref/fate/jpeg2000dec-p1_03         |   6 ++
> >> tests/ref/fate/jpeg2000dec-p1_04         |   6 ++
> >> tests/ref/fate/jpeg2000dec-p1_05         |   6 ++
> >> tests/ref/fate/jpeg2000dec-p1_06         |   6 ++
> >> 41 files changed, 361 insertions(+), 1 deletion(-)
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_hm_15_b8
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_02_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_02_b12
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_03_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_03_b14
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_04_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_04_b12
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_05_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_05_b12
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_07_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_07_b15
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_07_b16
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_08_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_08_b15
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_08_b16
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_09_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_10_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_11_b10
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_12_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_14_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_15_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_15_b14
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds0_ht_16_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_01_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_01_b12
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_02_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_02_b12
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_03_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_03_b12
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_04_b9
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_05_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-ds1_ht_06_b11
> >> create mode 100644 tests/ref/fate/jpeg2000dec-hifi_ht1_02
> >> create mode 100644 tests/ref/fate/jpeg2000dec-hifi_p1_02
> >> create mode 100644 tests/ref/fate/jpeg2000dec-p1_01
> >> create mode 100644 tests/ref/fate/jpeg2000dec-p1_02
> >> create mode 100644 tests/ref/fate/jpeg2000dec-p1_03
> >> create mode 100644 tests/ref/fate/jpeg2000dec-p1_04
> >> create mode 100644 tests/ref/fate/jpeg2000dec-p1_05
> >> create mode 100644 tests/ref/fate/jpeg2000dec-p1_06
> >
> > on linux x86-32
> >
> > --- tests/ref/fate/jpeg2000dec-p1_04 2024-11-10 01:23:28.599497603 +0100
> > +++ tests/data/fate/jpeg2000dec-p1_04 2024-11-10 02:38:03.497270241 +0100
> > @@ -3,4 +3,4 @@
> > #codec_id 0: rawvideo
> > #dimensions 0: 1024x1024
> > #sar 0: 0/1
> > -0,          0,          0,        1,  2097152, 0x9df478aa
> > +0,          0,          0,        1,  2097152, 0x5b4dd999
> > Test jpeg2000dec-p1_04 failed. Look at tests/data/fate/jpeg2000dec-p1_04.err for details.
> > make: *** [tests/Makefile:310: fate-jpeg2000dec-p1_04] Error 1
> > make: *** Waiting for unfinished jobs....
> > --- tests/ref/fate/jpeg2000dec-ds1_ht_04_b9 2024-11-10 01:23:28.599497603 +0100
> > +++ tests/data/fate/jpeg2000dec-ds1_ht_04_b9 2024-11-10 02:38:03.529270529 +0100
> > @@ -3,4 +3,4 @@
> > #codec_id 0: rawvideo
> > #dimensions 0: 1024x1024
> > #sar 0: 0/1
> > -0,          0,          0,        1,  2097152, 0x9df478aa
> > +0,          0,          0,        1,  2097152, 0x5b4dd999
> > Test jpeg2000dec-ds1_ht_04_b9 failed. Look at tests/data/fate/jpeg2000dec-ds1_ht_04_b9.err for details.
> > make: *** [tests/Makefile:310: fate-jpeg2000dec-ds1_ht_04_b9] Error 1
> >
> > thx
> >
> > [...]
> > --
> > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >
> > Modern terrorism, a quick summary: Need oil, start war with country that
> > has oil, kill hundread thousand in war. Let country fall into chaos,
> > be surprised about raise of fundamantalists. Drop more bombs, kill more
> > people, be surprised about them taking revenge and drop even more bombs
> > and strip your own citizens of their rights and freedoms. to be continued
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".

More information about the ffmpeg-devel mailing list