Re: Fwd: wip/new-model review

[Colin Walters <walters verbum org>]

On Thu, 2013-07-18 at 15:43 -0400, Jasper St. Pierre wrote:

> I think this is wrong. Every single task, including from resolve to
> zdiff, knows about a general concept of a "build". 

Ok, that's possible.

> The thing that I want to do is remove this central lookaside of
> "builds" that everything pulls from, and require the developer
> commands to specify which build they're working on.

I guess I'd envisoned entirely separate directories for multiple builds.
Something like:

~/builds/ostbuild/buildmaster/builds/20130719.0/
~/builds/ostbuild/buildmaster/builds/20130719.1/
~/builds/ostbuild/3.10/builds/20130710.5/
~/builds/ostbuild/tmp-bug728719/builds/20130719.0

And then we have one explicit toplevel cache directory:
~/builds/ostbuild-cache/

And one explicit toplevel source directory:
~/builds/ostbuild-src

This is like how OpenEmbedded's bitbake works; the cache is SSTATE_DIR,
and the src is DL_DIR.

One neat thing that bitbake can do (as well as Baserock) is pull cached
binaries from a network server.

Now at least bitbake will default to using SSTATE_DIR/DL_DIR inside your
build directory.   Maybe this is wrong through, and we should emphasize
branching a bit more.  So the developer flow could look like this:

$ ostbuild make-build-directory ~/build/ostbuild
  # makes ~/build/ostbuild/{src,cache,repo}
$ cd ~/build/ostbuild
$ ostbuild make-build buildmaster
  # Copies in or symlinks manifest.json

Then as much as you want:

$ cd ~/build/ostbuild/buildmaster
$ ostbuild build
  # Reads/writes to both ~/build/ostbuild/{src,cache,repo} as well
  # as ~/build/ostbuild/buildmaster.
$ <hack hack hask>
repeat

The fundamental difference between this and your proposal is that you'd
have me type:

$ ostbuild build --from=buildmaster
$ <hack hack hack>
repeat.

But typing the --from seems like it'd get annoying fast...

> I want to basically take "build management" out of the tasks and into
> the invoker -- in this case, the autobuilder has its own idea of how
> to manage these builds (with the builds/, tasks/, results/ structure
> we talked about in-person, which I should probably write up somewhere)

Yeah, need that written down.  Specifically what happens to say the
repo?  If we split that into separate directories, then we lose the
advantage of content-addressed storage between the builds, and branches
would no longer be cheap.

> The issue with "resolve" is that it's not really part of a build. The
> only essential part of a build is the snapshot, and in the new model,
> it's the "id" of a build. If you have the snapshot, you have a
> complete description of a build, and can remake the rest of everything
> at any time. (ignoring things like network status, repositories moved,
> build dates in binaries, etc.)

Right.

> Right now, the autobuilder creates a skeleton dir, runs resolve, and
> if resolve says nothing changed, we remove the directory and don't
> bump the build number.

I definitely like that.

> Additionally, for disk space / network optimization reasons, we keep
> the git repos outside of a build, and symlink them into the build when
> the autobuilder creates it, along with the other build skeleton files
> (last build, ostree repo).

Ok right, so I guess what you're saying here is things are shared, but
it's kind of hidden because we symlink them.

But the problem with that is it doesn't solve the concurrency issue.
Like what happens if two "builddisks" try to run at the same time and
follow the symlink to the last-build qcow2 disk?  Then you have two
kernels writing to the same ext4 filesystem, and disaster will result.

Now OSTree is robust against both concurrent reading and concurrent
"adders".  However, to delete data from a repo (ostree prune), all
readers and writers need to be offline at the moment.

To handle this correctly, something like:

$ ostbuild make-build bug741328

would look in ~/ostbuild/cache/disks, lock out any readers, and do a
*copy* of the file.  Likewise we'd need to atomically "cp -al" the
bulidroots and other cache data.

Or hmm.  Maybe we have a model where starting a build "steals" disks
from the cache, and then puts them back when done.

> I thought passing "last-build" in was fine, but for the other two, we
> should probably just load everything from a config file, similar to
> jhbuildrc (~/.ostbuildrc?)

Mmmm...that's the way it used to be, since I was modeling ostbuild after
jhbuild, but I switched to the bitbake model of "build directory is
context".  It's really been a lot saner.