Re: [Evolution-hackers] Distcheck passes !

[Tristan Van Berkom <tristanvb openismus com>]

On Mon, 2013-03-18 at 08:40 +0100, Milan Crha wrote:
> On Sun, 2013-03-17 at 22:00 +0900, Tristan Van Berkom wrote:
> > After running distcheck over and over, finding and fixing
> > several issues, I was finally able to pass it :)
>
>       Hi,
> thanks for all the work on this. One question, was it about "fixing" our
> tests, which were working properly before 3.6, or by fixing the cause of
> the test breakages?

A little bit of three:

  a.) I had to disable some tests, they are now in BROKEN_TESTS variable
      and should be fixed in some way.

      These include:
          tests/libecal/test-ecal-get-free-busy
          tests/libecal/test-client-get-free-busy
          tests/libecal/test-client-get-revision

      The free-busy API needs a rework, it just cannot work as defined
      as we've discussed at length on IRC, so I've disabled it.

      The test-client-get-revision is currently broken and should be
      re-enabled once https://bugzilla.gnome.org/show_bug.cgi?id=693464
      is fixed.

      I added some comments on the bug, basically I think it's not
      correct that property notifications should be pushed over D-Bus
      in this case, at least for some properties they need to be
      explicitly fetched over D-Bus (otherwise client/server cannot
      be properly synchronized)... Perhaps D-Bus offers a method
      to synchronously "re-sync all local properties" in a proxy,
      which could be used... I don't know.

      Another test I disabled you will notice while compiling, one of
      the cases in tests/libebook-contacts/test-untyped-phones.c is
      broken. I intentionally let the test case be 'defined but not
      used', hoping that people will keep being annoyed with this
      compiler warning until the code is fixed to meet the test
      expectations (or the test fixed to expect what actually happens).

  b.) Tried to address problems in EDS in order to get tests passing,
      notably the bug I already mentioned, test-client-self.

      While I did spend some effort here, the race is still not fixed.

  c.) Various build related issues needed fixing, for instance the
      docs/reference/libebackend directory was demanding that the
      gtk-doc make check target be enabled.

      But documentation in EDS is not of good quality enough to be
      listed as a prerequisite of distcheck, so I disabled make check
      in the documentation.

      Another issue of this type was that I needed to add a target to
      properly 'make clean' in the test directory (removing the test
      cache where addressbooks and EDS resources are created during
      test execution).

> The difference is obvious, the pre-3.6 tests made sure that most common
> usages run properly, thus if there were "fixes" in tests to make them
> run in 3.7.x, then it might imply that each application which uses
> the EClient (or any other eds API) in a similar way as the tests might
> be reviewed and retested for functionality, and possibly adapted in
> the similar way as our own tests.

Of course, I tried to be as less intrusive as possible and reuse all
the same expectations and code from older test cases.

But understand, in 3.7.x ALL test cases needed to be rewritten, to
use the new fixture and function in an isolated environment.

Some other tests were removed (such as a test which searches command
line arguments in the addressbook), because those expect an installed
EDS environment to test against (search cases should all be handled
by test-client-custom-summary now though, which is very thorough for
many types of query).

Cheers,
      -Tristan