Re: [BuildStream] Please try tox ! [Was: Update CI and testing process]

[Tristan Van Berkom <tristan vanberkom codethink co uk>]

On Fri, 2019-01-11 at 16:04 +0000, Angelos Evripiotis wrote:
> Hi Tristan and Chandan,
[...]
> Just wanted to say that the change I've noticed is that this is awesome,

:)

> I find it much simpler and faster to run tests and build the docs.
>
> After fiddling a bit, I've found that I can run a single test in just a few
> seconds and ~30 lines of output like so:
>
>     tox -e py36 --develop -- --no-cov
> tests/frontend/workspace.py::test_open[git]
>
> This is good enough that I don't feel the need to invoke pytest separately
> anymore, much better!
>
> I've added a hint about '--no-cov' to contributing.rst here:
> https://gitlab.com/BuildStream/buildstream/merge_requests/1059
>
> I don't know enough about Python packaging to make a recommendation
> about '--develop' or 'skipsdist=True'. Maybe I'll come back to that
> another time, I find it saves a few seconds for our project:
> https://tox.readthedocs.io/en/latest/example/general.html#avoiding-expensive-sdist

Well, we removed a lot of cruft from `.gitlab-ci.yml` expressely
*because* tox takes cares of creating and installing a package for us,
i.e. the intention is to run all tests from an installed package and
that it ensures at all times that the package has everything it needs.

We could probably make it not install a package by default and only do
it in CI... but I think it will be a challenge to ensure that coverage
information is collected properly in every scenario.

Cheers,
    -Tristan