Re: linux-user-chroot: Mounting more stuff inside the chroot

[Colin Walters <walters verbum org>]

Hi,

On Wed, Jun 3, 2015, at 06:56 AM, Sam Thursfield wrote:

> I've been looking closely at linux-user-chroot recently (and I think 
> ostree-list is the best place to discuss its development). I'm working 
> on Baserock, so my concern is with creating reproducible sandboxes for 
> building and deployment. The Baserock build tooling has always used 
> linux-user-chroot but up til now has required that builds are run as 
> 'root' anyway. I'm now looking at making builds work for non-root users.

Makes sense.

> On the linux-user-chroot side of things, this means making it possible 
> to mount more stuff inside the chroot. Previously we would call 'mount' 
> in the build tool, but that obviously requires 'root'. My first step was 
> to allow mounting a tmpfs at /dev/shm. Attached is a patch that adds a 
> --mount-tmpfs option, similar to --mount-proc.

Hm...do you gain anything here over:
tempdir=$(mktemp -d /run/user/$(id -u)/l-u-c.shm.XXXXXXXXXX)
linux-user-chroot --mount-bind $(tempdir) /dev/shm

The advantages of this approach are that the storage space is under
space already writable by the user.  I'm not aware right now
of any support for quotas on tmpfs, but were it to be implemented,
it would transparently Just Work with the logged in session, whereas
if l-u-c mounted a tmpfs, the accounting to the uid would be lost.

We could take the code to do the mkdtemp inside l-u-c, but since
it can be done outside of it, it seems like something better to just
document.

> One option is to bind-mount /dev from the host, which is already 
> possible with --mount-bind. This seems like a hole in build 
> reproducibility, though: a ./configure script could conceivably change 
> its behaviour based on the presence of a file in /dev, or something. I 
> really want to control what is in /dev more tightly.

Continuous uses the host's /dev; I'm a bit sceptical that there's
any (reasonable) program that would be attempting to access the host physical
devices to *build*.  Aside from /dev/{u,}random of course, which
you already have to think about how to handle for reproducibility.

> One idea I had was to add a new option like '--standard-mounts' or 
> '--mount-systemd-container-interface' to linux-user-chroot that would do 
> exactly what systemd-nspawn does. It seems good to be aligned with an 
> existing spec.

The above said, I'd definitely take a patch for the above, it should
be pretty simple and safe.  I *do* like the idea that I could cut off
access to e.g. the GPU from software I'm building, even if udev
has granted my id access to `/dev/dri/card0`.

> While I like that idea, it doesn't quite solve the immediate problem for 
> me as currently Baserock definitions are allowed to define arbitrary 
> device nodes. This is actually only used in one place, as far as I know: 
> <http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/fhs-dirs.git/tree/fhs-dirs.morph#n15>. 
> Static device files seem like a thing of the past so I'd like to just 
> get rid of this feature 

Yeah, I'd get rid of it; specifically devtmpfs obviates it.

> I'd like to hear thoughts on the changes I've proposed above, especially 
> which changes you think would be generally useful if added to 
> linux-user-chroot, and which changes sound like hacks that we might as 
> well keep to ourselves.

Though before we add *too* much more we should probably look
around and see whether there's any other projects that are doing
something similar that we could use instead.

There's a ton of container tooling out there.

Most other container frameworks like systemd-nspawn, Docker,
rocket all support running code as uid 0, which dramatically
increases the attack surface.

Sandstorm.io and xdg-app are conceptually closer to l-u-c,
although they both do a lot more..Sandstorm.io in particular
has a ton of functionality, and it's not really designed for
use by build systems.

One thing both of those do is use seccomp, which would also
make a lot of sense for l-u-c.

I should probably talk to Alex about sharing some code with
xdg-app for things like the seccomp blacklist.

So...I guess the conclusion is let's keep hacking on l-u-c but
keep pretty conservative about feature additions?