Re: Adding Avahi and USB drive support to libostree

[Colin Walters <walters verbum org>]

On Thu, Mar 30, 2017, at 08:03 PM, Philip Withnall wrote:
> How rigorous is the structure for naming refs? 

One can't *parse* them as there's no requirement for where
e.g. the architecture lives, or whether the ref even has one.
But I think we've established a precedent that ensures they're
sufficiently unique.

That said...there are some deep questions here, like for the
`org.gnome.Calculator/x86_64/stable` flatpak, what happens
if e.g. Endless ships a flatpak of it, and then one
adds the GNOME upstream remote?  (Or vice versa)

Now, we could try to say that only flatpaks delivered via
GNOME can be org.gnome but without an enforcement mechanism
we're certainly going to end up in situations where there
are some conflicts.

(Right?  Actually, I need to dive into this part of flatpak more...)

> Would it be possible to
> use the common prefix of the refs as an advert? That would avoid having
> to generate some kind of UUID for the repository, but should also mean
> the DNS-SD records are relatively small, assuming that each repository
> typically contains high-related refs which share a common prefix.

Perhaps we could advertise a [Bloom filter](https://en.wikipedia.org/wiki/Bloom_filter)
of the set of refs?

> In that case, the process for working out which peers to download from
> would be (ignoring internet and locally-mounted repositories for the
...

Yep, agree with 1..5 there!

> Step 4 could be optimised by including the commit timestamp as
> additional metadata for each ref in the summary file, so we can order
> the commit checksums before downloading them. I don't know if summary
> files routinely contain these timestamps already (it doesn't look like
> the EOS ones do).

Yeah; we can add metadata to the summary file but - my instincts
are telling me that what we have in 1..5 even without introducing
bloom filters is going to work fine on e.g. an 802.11b wireless network
with 30 machines.  

> That would work well, and would complement what I put in step 2 about
> having the summary timestamp in a DNS-SD record. 

I think we may want both the timestamp of the summary *and* the
timestamp of the newest commit.  But that said, again I'd
like to get to a first pass we know works and will be free of any embarassingly bad flaws,
and revisit later optimizations based on more real-world results.

> Ah, that would indeed speed things up in the case where the same commit
> checksum's metadata could be downloaded from several places. Am I
> missing any other situations it would speed up?

Offhand, I can't think of any.

> I'm thinking of roughly 30 peers on a local network, in a kind of
> classroom situation. We should probably design for a larger network
> though, to make things a bit more robust. Sorry if I didn't make it
> clear before: I don't think the cost of downloading the summary file is
> too much. The cost of having large DNS-SD records is too much.

OK, makes sense.

> I'm not sold on the idea of an expected update time. That sounds like
> the kind of policy decision which should be made at a higher level in
> the stack (like gnome-software). When libostree is asked to resolve a
> repository to pull from, I think it should make a best effort at doing
> that.

Right.  The reason I mentioned this is I'd like to standardize an
attribute for this, since at least on the Fedora/CentOS/RHEL side
we have regular cadences, and it'd be useful to show in the UI.

> Yeah, definitely. One more thing to note is that DNS-SD records are not
> secure ...

Right, but if you have a hostile machine on the same local network, they
can do lots of ugly other stuff like DNS/DHCP spoofing too unless
the network is specifically configured to prevent it.

> Hopefully I've explained adequately above. I think some of the
> confusion might have stemmed from me not quite remembering exactly how
> metadata is split between the summary and commit metadata files.
> Hopefully the step-by-step example above of what I have in mind is a)
> correct and b) clear. :-)

Yep, I think we're on the same page!