Re: tests/ and feature tests reorganized

[Stefan Westerfeld <stefan space twc de>]

   Hi!

On Mon, Oct 16, 2017 at 01:10:13AM +0200, Tim Janik wrote:
> I've just moved tests/ to use a single non-recursive makefile.
> In the process, all audio feature tests got moved into af-tests/
> also using a non-recursive makefile.
>
> This means all test from tests/ and af-tests/ are now executed
> in parallel, plus the resampler tests also became more
> parallelized by splitting up the test rules.
>
> Since the tests run a lot quicker on multi-core computers now
> (I could save more than 5/10 seconds for test runs here), I've
> added the af-tests back into 'make check' with a few exceptions:
>
> 1) Partymonster is not enabled by default. Though it passes,
>   it blocks the CPU for a solid 7 seconds on my laptop, while
>   all other tests complete in half the time. I'd love to add
>   it in though, but it needs to be faster, what about cutting
>   its rendering to a few seconds? Maybe 60 seconds or so
>   into the song?

Yes, maybe it should be a full test if you call make slowcheck (do we still
have a slow variant of make check?) or whatever, and check the first minute
otherwise, which sounds like a reasonable thing to do.

> 2) A bunch of tests are never completing now, they keep rendering
>    into the WAV file forever, filling up the disc. The culprits
>    are commented out for now:
> # FIXME: check-af-tests: af-tests/artscompressor
> # FIXME: check-af-tests: af-tests/minisong
> # FIXME: check-af-tests: af-tests/organsong
> # FIXME: check-af-tests: af-tests/simple-loop
>
> Not sure I'll get around to fixing this soon,
> but it definitley must be addressed.

Definitely.

> Also, it could really help to fix up resamplertest and bsefcompare
> to *only* put out '  PASS       blah short text blah' unless -v is
> given or a test is failing. That way the mixed-line output of
> parallel test runs becomes parsable again.

Ok, I found some time to implement one line test output, the merge
request for that is

  https://github.com/tim-janik/beast/pull/23

   Cu... Stefan
-- 
Stefan Westerfeld, http://space.twc.de/~stefan