Jenkins-based Continuous

[Vadim Rutkovsky <vrutkovs redhat com>]

Hi,

I was playing with Jenkins and Continuous recently and got some interesting results, which might improve 
current situation with UI. 
Jenkins takes care of many minor aspects current http://build.gnome.org is missing like storing task history, 
many plugins, configuration without committing to git etc.
The heart of it is still the same 'ostbuild make', but with different params and overall workflow.

The workflow on Jenkins will change a bit:
 1. Resolve task is time based - same as current implementation
 2. Instead of a building all components resolve task will put a build for each component in a queue.
 3. Each component is build built separately. After each build the task builds a runtime image and runs 
smoketests. Later on we'll be able to revert ostree commit if smoketest has failed.
 4. Resolve task also puts 'build -devel-debug and -hwtest' task in the end of the queue, which will trigger 
Installed Tests and Applications test
 5. As all component tasks are scripted, any commit to gnome-continuous repo will reconfigure tasks 
automatically - this would allow seamless tagging component and their adding/removal.

Here are the biggest advantages in this scheme:
 1. If one component fails to build the tasks won't get stopped
 2. The build queue clearly shows what tasks are happening and about to happen, along with time metrics, git 
changes and direct console output
 3. Admins can also force building the component or quickly 
 4. Simple script converts application and integration test results in JUnit XML, so Jenkins allows us to 
track the status and history for each testcase.

Some potential improvements here are:
 1. A wide range of notifications: custom emails \ IRC etc.
 2. Test builds from different branch or patches from bugzilla.
 3. Filtering out translation updates (or setting less priority to those)
 4. Trigger other events after build is complete: e.g. start a build on perf.gnome.org
 5. As Jenkins is a proper systemd/upstart service, this will finally fix a sticky situation with server 
startup
 6. We can run custom scripts / static analysis tools for each component: e.g. cppcheck or pep8 for python 
projects of requested.

Biggest disadvantages are:
 1. Java. Builders should be able to run java client to execute instructions 
 2. Integration/Application tests and -devel-debug are being run less frequently
 3. Web UI might look a bit more complicated for beginners


In a short discussion Colin's main consideration was security. I guess we'd better set anonymous users to 
read-only (and even hide some jobs if needed) and create accounts for the interested people (or use gnome's 
LDAP data if available).

Unfortunately I didn't manage to setup a public available instance (available for Red Hat employees, 
however), but I think I can provide some screenshots / detailed information if requested.

Thanks,
  Vadim