Re: Some analysis on live.gnome.org performance

[Owen Taylor <otaylor redhat com>]

On Thu, 2009-12-10 at 10:30 -0800, Jeff Schroeder wrote:

> > Possible fixes:
> >
> >  - Block /TitleIndex and /WordIndex entirely - they aren't useful pages
> >  - Block the Blue Coat fetches by User Agent (this, however, apparently
> >   doesn't get all the prefetches, sometimes it uses the user agent
> >   of the requesting client.)
> >  - Use apache's mod_cache facilities to cache /TitleIndex, /WordIndex
> >  - Patch Moin to omit this section of the pages
> >
> > Don't have a lot of opinion which one of these or combination of these
> > is best - the last one makes some sense to me.
> >
> > - Owen
> 
> Sorry Owen I forgot to reply all the first time.
> 
> The last one makes a lot of sense however it will require updating the
> patch as we upgrade moinmoin. What are the downsides of just blocking
> both of those URLS with a shiney gnome 403 page? Besides it being
> nifty to see those pages, is there any value add in keeping them?

Downsides I could see:

 - These pages are linked to from http://live.gnome.org/HelpForBeginners
   and might have some small utility

 - Just blocking the /TitleIndex and /WordIndex won't keep Blue Coat
   from predictively scraping other URLs in that section.

   From a rough grep, 10% of the page hits on live.gnome.org are 
   for action=raw or action=print. 

   (Since there is no UI for getting to action=raw or action=print
   I can find, we could also possibly just block those as well.)

- Owen