[FFmpeg-devel] [PATCH v2] fate: Allow running multiple rounds of tests with differing settings

[Martin Storsjö]

On Mon, 18 Dec 2023, Martin Storsjö wrote:

> This can be used to run tests multple times, with e.g. differing
> QEMU settings, by adding something like this to the FATE configuration
> file:
>
>    target_exec="qemu-aarch64-static"
>    fate_targets="fate-checkasm fate-cpu"
>
>    fate_environments="sve128 sve256 sve512"
>    sve128_env="QEMU_CPU=max,sve128=on"
>    sve256_env="QEMU_CPU=max,sve256=on"
>    sve512_env="QEMU_CPU=max,sve512=on"
>
> It's also possible to customize the target_exec command further
> by injecting a sufficiently quoted variable into it, which then can
> be updated for each run, e.g. target_exec="\$(CUR_EXEC_CMD)".
>
> For each of the environment names in fate_environments, the tests
> that are run get the name suffixed on the fate tests in the
> test log and fate report, e.g. "fate-checkasm-h264dsp_sve128".
> ---
> v2: Added examples to fate_config.sh.template.
>
> Will push this together with
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231127123118.3622784-1-martin@martin.st/
> soon if this seems reasonable.

Pushed now.

// Martin