[FFmpeg-devel] [PATCH] fate.sh: Allow overriding what targets to make for running the tests

[Rémi Denis-Courmont]

Le maanantaina 27. marraskuuta 2023, 14.31.18 EET Martin Storsjö a écrit :
> This can be useful if doing testing of uncommon CPU extensions by
> running tests with QEMU (by configuring with e.g.
> "target_exec=qemu-aarch64"), by only running the checkasm tests,
> to get a reasonable test coverage without excessive test runtime.

For the purpose of testing future or bleeding-edge CPU extensions on emulator, 
you would normally want to be able to actually filter those in. That is more of 
a matter of patching checkasm than FATE.

Considering the poor coverage of checkasm, I fear that this just gives the 
wrong impression, not to say a false sense of security. It feels misleading to 
encourage or support that paradigm into FATE, in light of that poor coverage. 
Afterall, if it's just about running checkasm, anybody can just run
`make tests/checkasm/checkasm && tests/checkasm/checkasm`.

Either way, this feels like a case of cart before horse.

Also FWIW, RV broke due to misaligned accesses and illegal vector types that 
QEMU tolerated. That is rather an argument against QEMU than against this MR 
but still.

-- 
Rémi Denis-Courmont
http://www.remlab.net/