Re: Adding Avahi and USB drive support to libostree

[Colin Walters <walters verbum org>]

On Wed, Mar 29, 2017, at 07:25 PM, Philip Withnall wrote:
> That makes sense. Using the ref name as an identifier would be a
> possibility, but I don't think it would work with Avahi, since a peer
> would have to advertise every ref it has,

Why?  Are you concerned about polling each remote for each ref?
If the remote has a summary file, then we're talking about one
HTTP request per remote.   If it doesn't, things indeed get a bit
uglier, but I suspect there are things we could do to optimize
that, like remembering which Avahi nodes had any content
we wanted before.

We currently have some code to cache the summary file which only
kicks in if they're signed, but we could tweak that, then we'd do
If-Modified-Since as well.

> Yup. Though sorting by priority doesn't help when you're downloading
> the commit object; it should be done once the commits have all been
> downloaded, to address the case where the most recent commit is
> available via a slow transport and a fast one.

We can definitely teach the fetcher code to prioritize file:// and Avahi
over not-Avahi http://.

> This can be sped up in the Avahi case by including the refspec and
> commit timestamp in DNS-SD records, so the summary file doesn't have to
> be download from all peers --- just the ones we choose to pull objects
> from. Though this would run into the oversized DNS-SD record problem I
> mentioned before.

How many peers are we talking about here?  Surely the cost of a HTTP
request for the summary file isn't going to be large.  We don't need to query *all*
of them either - one optimization to make here is to include an "expected update time"
in the commit metadata.  If the current time is *before* that, we might only query
one or two peers once a day.  If it's *after* that, we could get increasingly more
aggressive in trying to find an update via Avahi?

I guess one thing that would help a lot too is to advertise the *timestamp* of
the summary file in the DNS-SD record. 

But again, I'd like to understand why you think the cost of querying each
remote for a ref would be too high.