Re: Pure rust ostree FUSE implementation

["Will Manley" <will williammanley net>]

On Tue, 18 Aug 2020, at 23:44, Colin Walters wrote:
> On Wed, Aug 12, 2020, at 12:51 PM, Will Manley wrote:
> > I've written a FUSE implementation which is backed by an ostree bare-user repo:
> >
> > https://github.com/wmanley/ostree-oxide
>
> Very interesting!  Feel free to send a PR linking to this from the main 
> page under related projects.
>
> Rust is certainly really good for FUSE.

Yeah, I found it quite straightforward once gvariant-rs was done thanks to fuse-rs. Getting it working just 
consisted of implementing the required trait, and the compiler and IDE integration guide you along how to get 
it right.  At this point I've applied zero discipline (WRT ostree-oxide, not gvariant-rs), so it's been plain 
sailing.  The next step would be to add some real tests, CI, etc.  If there's interest I may be able to 
summon the motivation :). 

On the subject of IDEs: I'll say that I found using VSCode and rust-analyser really helpful.  I've never 
really used an IDE before.  Previously I'd found that IDEs get in the way and are just not very useful while 
making development feel sluggish and unpredictable.  Maybe it's because I've mostly been writing C++ and 
Python, both of which defy useful and timely analysis, so I'd written off these kinds off tools as toys. But 
the interactive experience of VSCode and rust-analyser has made me a convert, at least when developing Rust. 

> > It's written in pure rust and doesn't link to libostree or glib.  
>
> Do you have a reason for that?  Memory safety?  Performance?  I have 
> thought in the past that avoiding some of the GInputStream abstractions 
> as well as using crossbeam/rayon for things like committing objects 
> would be a huge speedup.

There's a few reasons here.  The biggest one is that I wanted to learn more Rust, and this seemed like a good 
project for that.  I already had a good understanding of OSTree and GVariant, so it seemed a useful learning 
exercise.

The other reason is to avoid a dependency on GLib.  GLib is really great as a C standard library and the free 
bindings to other languages are really helpful, but it does come at a cost.  It defines an OOP object model 
with liberal use of inheritance, reference counting and allocations that IMO aren't a natural fit for rust 
projects.  It also complicates building - without it you don't need to integrate C projects/autotools/make 
with the build process, we can just use cargo.

GLib also implements its own cross-platform abstractions which duplicate those that are available in Rust.  
This has a learnability overhead because even if you already understand things on a Linux syscall level, you 
also need to understand the GLib model built on-top.  Specifically: I've feel that I've got a good 
understanding of the underlying ostree data model including merkle trees and how that model maps onto UNIX 
concepts like inodes, hard-links, filenames, etc.  I find it difficult taking that understanding and mapping 
it to the `GFile` interface and reasoning about what syscalls a particular function call will call.

My experience when working on #1643 was that it was actually easier for me to write the code that iterated 
through the dirtrees, using `ostree_repo_load_variant` and the GVariant APIs than it was to go through the 
ostensibly higher level GFile based API.  The code came out a lot faster too.  Since following rust for quite 
a while I kept thinking about how much easier it might have been to do and make robust with Rust.

> But...reimplementing everything there is not a trivial endeavor.

Indeed.  Ostree is quite intricate as it needs to integrate with all these external systems and handle a 
whole bunch of different use-cases that happen when software collides with the real world.  But - at its core 
- the model is very simple.  For a FUSE filesystem we don't need to worry about interactions with 
bootloaders, SELinux xattrs, pushing, pulling, gpg, etc.  We only need to know how to interpret an ostree 
repo, which is fairly straightforward.

Further to this: if we were to further oxidize ostree itself I think it makes sense to start in the middle 
and work outwards. What I mean is: The API is a C/GLib API and cannot change.  So to oxidize incrementally 
we'll need C calling Rust (via some internal C API that Rust exposes).  I think it's simpler if rust doesn't 
also need to call back into C.  Otherwise the build could get quite complicated WRT calling the bindings 
generator at the appropriate time.  I'd imagine that this was the motivation around oxidizing bupsplit first 
- it's self-contained, exposes a narrow API of plain C types and doesn't require calling back into C.

> > https://docs.rs/gvariant/0.3.0/gvariant/
>
> Nice.  Have you compared with https://lib.rs/crates/zvariant ?

Yes.  At this point zvariant implements the DBus serialisation format, although I understand that it will 
support GVariant in the future and the GVariant implementation has made good progress.

I'm no expert in zvariant, so take any opinions here with a pinch of salt...

My library makes some different design decisions to zvariant. zvariant derives the GVariant types from the 
Rust types you give it.  gvariant-rs requires that these types are explicitly called out in GVariant format.  
I think this is the key difference in approach.  IMO data takes priority over code, as it's easy to change 
code, but hard to change data that's been saved on a disk somewhere - so I always like serialisation formats 
to be explicit.

zvariant integrates with serde. This is definitely a useful feature. If I were to add serde support to 
gvariant-rs I'd do it in a different crate.  I think in this way gvariant-rs is a lower-level API than 
zvariant.

gvariant-rs doesn't implement the DBus message format, zvariant does.  I think this is probably irrelevant 
for ostree purposes.

gvariant-rs makes allocations explicit and can operate in no-std and no-alloc environments. It's designed for 
reinterpreting byte buffers as GVariant types for fast and light access - as such control over allocations 
and alignment is exposed in the API, while zvariant doesn't expose such control.

My intention is to extend the GVariant schema compiler[1] with a backend for Rust using gvariant-rs.

[1]: https://blogs.gnome.org/alexl/2020/01/14/introducing-gvariant-schemas/

> (GVariant is a superset of DBus...I think it boils down to the "maybe" 
> type only existing in GVariant)

To be specific: The GVariant data model is a superset of DBus' in that you can round trip DBus messages to 
GVariant and back losslessly, but the serialisation format is different.  GVariant, unlike DBus, supports 
O(1) array and struct element access.

> There's probably a better forum for this...it looks like the gtk-rs 
> developers use Matrix.  Or we could try to summon the zvariant 
> developers here.

I've had a few conversations with @zeenix (the zvariant author) on github issues. I wouldn't want to put 
words in his mouth, but I sense a degree of frustration that I chose to implement my own GVariant crate 
rather than adding GVariant support to zvariant.  Ultimately I wanted to explore the design space, and 
gvariant-rs is what I made.  Creating it was educational for me and the intention is that it will hopefully 
be useful for others, in particular for ostree.  It was designed with ostree in mind - in particular making 
changes like #1643 or #1408 easier.

> On this topic I intend to build on top of the recent momentum in e.g.
> https://github.com/ostreedev/ostree/pull/2127
> for both more tests, 

Yeah, that's really neat.  How have you been finding it?

> but another thing I'd like to do is create an
> ostree-ext-rs crate that is in the ostree git repo, that is 
> "somewhat API slushy helpers on top of the ostree-rs crate",
> which would make it way easier for us to share code
> between ostree/rpm-ostree, and would also be useful for any
> other projects that are in Rust and using ostree which...I
> thought https://github.com/advancedtelematic/aktualizr was
> but looks like it's fully C++ now (or maybe I'm confused about
> the history).

I see.  What kinds of things would you want to accomplish with those APIs?

I hadn't realized that rpm-ostree already had rust in it.  Is it using the ostree-rs bindings?  I can't see 
it listed in your Cargo.toml.

Thanks for your comments

Will