Re: Status of Buildstream tests?

[Jonathan Maw <jonathan maw codethink co uk>]

On 2017-07-28 12:38, Justin Erenkrantz wrote:
> Hi!
>
> It seems that the tests from buildstream-tests are failing for me due
> to permission errors - like so:
>
> ---
> Error: File permissions differ for files results//tar-test/test
> (-rwxr--r--) and expected//tar-test/test (-rw-r--r--\n)Error: File
> permissions differ for files 'results//tar-test/test' and
> 'expected//tar-test/test'
>
> tar-test                              failed
>
> Tests for 'tar-test' failed.----
>
> This might be due to wonkiness due to checkouts from git losing
> permissions?  Should we have a script to reset the permissions to a
> known-good state?  Or?

Hi Justin,

Having a look at this, the issue is further confused by the way tar file 
permissions are being handled.
Looking at the tar that the file comes from, it has permissions 
-rwxr-xr-x.

In the long-term, we probably want the tar implementation to preserve 
permissions.
In the short-term, I think the simplest solution is to make this test 
disregard permissions when comparing files.

> Also, it seems that the tests aren't necessarily idempotent.  If I run
> a test twice, I get:
>
> ---
> ERROR: Checkout directory is not empty: results//tar-test
>
> Tests for 'tar-test' failed.
>
> ---
>
> Should we be deleting the results directory in the beginning of the
> test run?

I think this would probably be a good idea, as I tend to forget to run 
`./run-test.sh clean` more often than I need to look at what the results 
were.