Re: [gnome-desktop-testing] Mago logging

[Eitan Isaacson <eitan ascender com>]

On Mon, Sep 7, 2009 at 12:52 PM, James Tatum <jtatum gmail com> wrote:

* Add a main index page linking to the test suites 

* Show test logs from previous mago invocations

Yup, we need those two. I think each page should have a summary of the tests/suites/runs it is listing, for example, a list of test suites should show which test suites failed or partially failed. Same thing with the previous invocations page. I think another directory level would be useful with a timestamp would be useful to achive this, for example, ~/.mago/200909082011. For ease of access, there should be  symbolic link "latest" that points to the latest run. This is how I set up the live cd testing.
 

* Provide a lot more detail for each individual test case

I like this idea, i don't like the wrapping idea. Here is how it could be implemented:
LDTPv2 already provides per-command logging, and it would be easy to provide an API that would allow inserting hooks for each LDTP command, and a hook that would be run just before an exception is thrown upon failure. Then all that mago needs to do is use those hooks for it's own logging purposes.

One big issue that I see is that if an LdtpExecutionError is raised in LDTP, it doesn't automatically mean an unexpected failure, it is common practice to do try/catch blocks for expected failures, a bad example:

while True:
 try:
  frame.click('bntOK')
 except LdtpExecutionError:
  sleep (1)
  continue
 break