[Tracker] Functional tests land on master

[Ivan Frade <ivan frade gmail com>]

Hi all,

 Finally some work on functional testing that was cooking up in a branch in gitorious is landing in master. What is this?

 It is a collection of programs written in python (and using unittest2). They run against the installed version of tracker, but modifying the env variables so it doesn't break the default database (and warranty that they run in a "clean" environment). Then they perform some actions over the dbus APIs and the file system and check that the results are what we expect. So, they test tracker "as a whole".

 The usual flow of these tests is:
1. Stop any tracker process running in the system
2. Set the env variables (XDG_DATA_HOME, XDG_CONFIG_HOME, ...) to a test location (usually under ~/.tracker-tests/)
3. Launch the required processes (store, miner... depending on the test)
4. Do the actual testing
5. Stop the test processes and unset the environment variables

 What are they testing right now?

* inserting/removing/updating data in the store
* Different sparql functions (fts, tracker specifics, coalesce, and so on) on the store
* Backup, restore and journal replay
* Crawling and monitoring in the miner-fs (right now only create/move/delete files... we need to add folder operations)
* FTS from the miner fs (creating/updating/deleting text files and checking that we can find them with FTS functions in the store)
* Extraction (now we can translate every bug like "tracker doesn't extract the X from 
this file Y" in a functional test adding the file (cutted down to few 
Kb) and a small text file in INI format (X.expected) with the expected 
metadata.
* Writeback (for our limited supported test cases)

 and few more things are pending review like virtual files and "mass storage" (mounting/umounting volumes)

What are these test useful for?

* To check that we don't break anything when modifying some internals in tracker
* To verify if a bug is really there: write a test replicating the problem, and later we can use it to check it is fixed, and keep it in the check list of the previous point
* To check if tracker works fine in certain platform/environment (for example when upgrading sqlite, libpng or any other external dependency)

 How to run them:
http://live.gnome.org/Tracker/Documentation/FunctionalTesting

 Hopefully more bug reports will be translated into test cases and we will keep them up to date and make them PASS in a reliable way on every release.  
 
 Regards,

Ivan