[FFmpeg-devel] [RFC] Lavfi test system

[Stefano Sabatini]

On date Monday 2010-03-08 03:17:11 +0100, Michael Niedermayer encoded:
> On Mon, Mar 08, 2010 at 02:36:14AM +0100, Stefano Sabatini wrote:
> > On date Sunday 2010-03-07 17:44:20 +0100, Michael Niedermayer encoded:
> > > On Sun, Mar 07, 2010 at 04:20:16PM +0100, Stefano Sabatini wrote:
> > > > On date Sunday 2010-03-07 11:18:33 +0100, Stefano Sabatini encoded:
> > > > > On date Thursday 2010-01-07 23:36:02 +0100, Stefano Sabatini encoded:
> > > [...]
> > > > Index: libavfilter-soc/ffmpeg/tests/lavfi-regression.sh
> > > > ===================================================================
> > > > --- libavfilter-soc.orig/ffmpeg/tests/lavfi-regression.sh	2010-03-07 16:15:22.000000000 +0100
> > > > +++ libavfilter-soc/ffmpeg/tests/lavfi-regression.sh	2010-03-07 16:16:43.000000000 +0100
> > > > @@ -20,6 +20,8 @@
> > > >      vfilters=$2
> > > >  
> > > >      if [ -n "$test" ] ; then
> > > > +        slicify_h=$(expr 8 + $(date +%N) % 24)
> > > > +        vfilters=slicify=$slicify_h,$vfilters
> > > >          do_video_encoding ${test_name}.nut "" "-vcodec rawvideo -vfilters $vfilters"
> > > >      fi
> > > >  }
> > > 
> > > this is a problem because its not reproduceable
> > > 
> > > an LCG started from a constant would be better
> > 
> > I believed the idea was to try different slicing at every run... what
> > am I missing?
> 
> we dont want spurious unreproduceable failures
> or other way around spurious unreproduceable successes in teh test wen
> they should fail
> it makes debuging hard ...

I agree but this way we are using only a limited subset of slice
values.

Another solution would be to iterate a few times each test with
different slice height values taken from a pseudo random
sequence. Would this be OK?

> > Regards.
> > -- 
> > FFmpeg = Frenzy and Fostering Miracolous Problematic Erroneous Guru
> 
> >  lavfi-regression.sh |   10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > b361118ffebf15ed75e0bac79f4aa27b4d20d371  randomize-lavfi-test-slice-sizes.patch
> > Index: libavfilter-soc/ffmpeg/tests/lavfi-regression.sh
> > ===================================================================
> > --- libavfilter-soc.orig/ffmpeg/tests/lavfi-regression.sh	2010-03-07 16:17:19.000000000 +0100
> > +++ libavfilter-soc/ffmpeg/tests/lavfi-regression.sh	2010-03-08 02:29:41.000000000 +0100
> > @@ -14,12 +14,22 @@
> >  rm -f "$logfile"
> >  rm -f "$benchfile"
> >  
> > +get_lcg()
> > +{
> > +    echo "65539 * $1 % (2^31-1)" | bc
> 
> where does this generator come from?

Uh an experiment of mine, I did some little tests on it and it looked
good enough for this application.

> and why dont you use the generator we use elsewhere in ffmpeg?

Do you mean av_lfg_get()? Are you suggesting that I should use a C
program (lfg_get.c) for generating such a sequence?
 
> also what should be used as random value is the scaled down value not the
> modulo something value like you do later.

I have to say that I lack the understanding of the theory behind,
please if you can give a very short explanation, I'm looking at the
code using av_lfg_get() and most of the times a modulo operation is
performed.

Regards.
-- 
FFmpeg = Fiendish & Fanciful Mastering Ponderous Erroneous God