Re: Thoughts on a new network repository mode: subtrees

[James Antill <james fedoraproject org>]

On Tue, 2014-10-14 at 09:03 -0400, Colin Walters wrote:

> It's quite inefficient when going from major version -> major version. 
> Concretely, I tried going from a Fedora 21 Docker host tree to rawhide,
> and it was ~11000 HTTP requests.  It did work, but yes, very slow.

 Have you retested that recently, and did you use thttpd or
apache-httpd? When I did tests with it the latency/overhead of doing one
connection per. file wasn't really noticeable (when we are doing 8
parallel connections, and no fsync).

 One problem was that the trivial_httpd library, really is trivial ...
and not very good. But as soon as I used thttpd instead it didn't fail,
and number of files wasn't a problem.
 It's been a month or three now, but I was testing with like 10 100MB
files in one repo. and 10k-50k files in another repo.