Re: [Gimp-developer] Jenkins infrastructure (Was: Jenkins tutorial)

On Mon, Feb 24, 2014 at 2:09 PM, scl <scl gplus gmail com> wrote:

Forwarded with Sam Gleske's permission:

On  24.2.2014 at 3:21 PM Sam Gleske wrote:

I'm not familiar with the shorthand "sth".  I can start running tests
against <> and begin to start ironing

out bugs which would need to be fixed in my program.  I've run into bugs
here and there running my crawler against large websites.  Right now
I've crawled and run against Drexel University IRT website using the
crawler which is ~70k links.  I'll do it for
<> too and then contribute a Job when I have it ready

(and for the other GIMP websites as well).

I don't know whether this could cause technical problems for
The reason why I wrote 'something stable for production use' is that I want
to avoid just them ;-)
Thus it would be good to hear our website administrators opinion to that.

Both the crawler and the tests following it support a --request-delay.
Requests can be arbitrarily delayed by 0.3 seconds (smaller or larger if
chosen) so as to mitigate impact of the website.  I won't run any tests
against the GIMP website until I get permission to do so.  Also, currently
my testing suite support saving crawl data in a JSON format.  The advantage
of this would be separating the crawl from the actual testing (if that's
desired later on).  Currently I mostly use it to debug my test suite
because crawling can take a bit of time so saving that data allows me to
avoid that step.

On Mon, Feb 24, 2014 at 2:09 PM, scl <scl gplus gmail com> wrote:

 Also, I'd like to note that
if you're not using git hooks now would be a good time to use them.
Rather than using Jenkins to constantly poll the SCM simply implement a
hook to launch a Jenkins job when a certain branch is committed to.

Currently we're polling SCM every few minutes. Adding a git hook to the
repository would surely need involvement of the GNOME administrators.
Is there a good reason to switch over from polling to hooking?

The advantage of hooks vs polling is that hooks are on demand.  i.e. a job
or process is only launched if there is an actual commit.  For the sake of
covering all topics I'll briefly describe polling.  Polling simply keeps
checking the state of the repository periodically (e.g. every 5 minutes).
If the state hasn't changed (no commits) then it does nothing.  If the
state has changed (i.e. commits have been made) then it executes based on
what was defined for the Job.

Hooks simply take less processing time.  On a smaller scale they save
energy because they're not constantly polling.  It's also lowering the
request load of the server serving the repository being polled.  It's not a
big deal if a server is serving a single repository or a few repositories.
However, when you have e.g. 10000 repositories on a server the polling
request load of each repository every few minutes starts to eat up the
server's ability to serve.  For smaller servers it's not the end of the
world to have polling because it is easier to implement.  I realize the
GIMP project may be only a small part of that ecosystem being served but it
is something worth considering.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]