Re: Making ostree robust wrt concurrent use

[Alexander Larsson <alexl redhat com>]

On fre, 2015-12-11 at 16:15 +0100, Alexander Larsson wrote:
> On fre, 2015-12-11 at 11:23 +0100, Alexander Larsson wrote:
> >  
> > 3) During the operation we naturally build up a topological sorting
> > of
> >    all the objects involved in the operation. For instance, during
> > a
> >    pull we do a recursive discovery of all objects that are not
> >    available in the local repo. We save this order, and then use
> > the
> >    reverse order when moving non-leaf (DIR_META & COMMIT) objects
> > into
> >    the repo, doing an fsync of the object directories whenever we
> > leave
> >    all children of some object have been moved.
>
> Actually, this doesn't work because if static deltas are in use we
> just
> always trust that they have all the objects (or they are listed as
> fallbacks), so we never build up the topological sort. I guess we
> need
> to manually do this at the end of the transaction.

Actually, it seems like all the code that writes via a transaction
verifies the entire set of dependencies up to whatever limit was
specified (by max-depth, by remote having limited history, commit-only, 
by limiting to subdir, etc). This means there is never a problem in
having an object, but not its depencencies in the repo.

This means we can simplify this quite a bit.

I think we still need some kind of guarantee though. For instance, we
don't want to update the ref file to point to a commit before we're
fully sure that all the objects that were involved in the transaction
has been commited to disk. So we need to fsync all the individual
objetct directories after the renames before writing the refs.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl redhat com            alexander larsson gmail com 
He's a gun-slinging sweet-toothed jungle king with no name. She's a 
violent foul-mouthed socialite who dreams of becoming Elvis. They fight 
crime!