Re: Pointing ftp.gnome.org somewhere (Re: master.gnome.org and FTP server restructurement)

[Owen Taylor <otaylor redhat com>]

Mattias Wadenstein <maswan acc umu se> writes:

> On 10 Mar 2001, Owen Taylor wrote:
> 
> >
> > Tomas Ogren <stric ing umu se> writes:
> >
> > > After discussing the matter with ftp.gnome.org == master.gnome.org and
> > > anonftp being disabled with Martin on irc, we came to the conclusion
> > > that a possible solution is to point ftp.gnome.org at ftp.acc.umu.se
> > > (which is ftp.se.debian.org right now too) in Umeå, Sweden.
> [snip]
> >
> > Hmmm, I'm pretty sure a majority of our traffic is currently
> > from North America.
> 
> A quick look at some statistics from before anonftp access was cut of
> shows that about half the traffic (perhaps a little more than half) came
> from North America. Of course, that is just estimates from the TLD
> summary, unresloved IPs could be anywhere and com isn't 100% north
> america. http://www.acc.umu.se/~maswan/util/tmp/xferstat/xferstats.html
> is where I have the statisics up now, it will probably be gone in a couple
> of weeks.

Those aren't really reliable statistics. gnome.org was dropped down to
2mb at about the end of January, and on Feb 6 I dropped the user limit
to 15, so you are looking at stats for:

 - A small userlimit
 - A very crammed connection

Also, I don't see how you are doing that computation - from your
stats we have:

 com: 21%
 unresolved: 17%
 net: 16%
 edu: 12%
 ja: 6.0%
 ca: 3.5
 
Beore we get to any European domains. Making the assumption that
com/net are all North American (not true, but not that far off,
really) and that unresolved domains are equally distributed
(my experience, they are tend to be heavily NA cable modems.)

We get: NA fraction = (21 + 16 + 12 + 3.5 + 1.5 + 0.5 / (100 - 17))

 66%

And a the European fraction of about 23%.

But that's pretty much neither here nor there.

> [snip FTP performance test]
> > So, quite decent transfer rates from North America.
> >
> > But from a good-network-citizen point of view, if most of our traffic
> > is from North America, it seems that we probably should try to have
> > ftp.gnome.org in North America.
> 
> Well, it isn't that much traffic to begin with for us, or our network.
> Assuming the xferlogs that I got from ftp.gnome.org are representative,

They aren't, though unfortunately, they are the best we have.
My guess is that on a ftp.gnome.org with plenty of bandwidth
and a user limit of 100 we'd see about 10G of traffic a day
steady-state with peaks of 20-30G or so around releases.

But that's a pure blind guess.

And don't ask me why so many people are downloading the source
tarballs... (and often old, obsolete source tarballs as well!)

> we've had single users in North America pulling much more in one day from
> us than the entire daily traffic from ftp.gnome.org.
> 
> Besides, most of the traffic across the transatlantic links is in the
> other direction. Except that from ftp.sunet.se (~100 gigs/day to USA). :)
> 
> > I think the first thing we need to do is to make sure all
> > interested mirrors have the current rsync password and are rsyncing,
> > and we have their contact addresses.
> >
> > Then we'll be better in a better position to evaluate
> >
> >  - How much spare bandwidth we have at the main gnome.org site
> 
> Having master.gnome.org on a different network connection than the public
> sites is probably a good idea, so that mirrors always can get in and
> update decently fast. This might not be an issue in normal operation, but
> with major releases this can be quite important.

I would agree - and I'd like to keep that connection open for 
CVS/bugzilla/etc as much as possible as well as FTP mirrors.
 
> >  - Whether there are interested mirrors in North America with
> >    comparable connectivity.
> 
> Sure. But I think that one important thing is that we can have this up and
> running today. And doing a pretty good job at it too. If a better
> candidate would turn up a couple of months later, it isn't all that hard
> to repoint the name, right?

Certainly very much a point.
                                        Owen