Re: libcurl vs libsoup

[Allan Gardner <allangardner fedoraproject org>]

On Fri, Jul 22, 2016 at 5:23 PM, Matthew Barnes <mbarnes redhat com> wrote:

On 07/22/2016 04:26 PM, Colin Walters wrote:

Hey so, I'd just like to put out a feeler among ostree's users:
Who would strongly prefer libcurl over libsoup?

The reason I'm asking is right now, my primary consumer (rpm-ostree)
uses libcurl indirectly via librepo, so it's linked in process anyways.

As a first step you might want to subsume SoupURI into libostree as something like OstreeURI.  Then you could convert OstreeFetcher (with OstreeURI arguments) to an interface with separate libsoup and libcurl backends.

I'm working on the fetcher/pull stuff for GSOC, so I thought I'd pop in here. The project, if you aren't aware, is to fetch in a separate (unprivileged) process. The verification will remain in the main process, so a hijacked fetcher can't just write arbitrary data. Ideally, it will be cryptographically impossible for an attacker to hijack the update process.

My implementation plan, which I kind of outlined in a few places but not really:
1. Write DBus "fetcher" helper with fetch_object, fetch_ref, etc. methods (almost done)
2. Move all libsoup code into said "helper" service (not really done, but I've done some refactoring which will make it easier)
3. Pull code forks the service, calls the DBus methods on an anonymous socket, and cleans up once the pull is finished. (nearly there, although Glib is... confusing)

4. Helper service becomes a separate executable (not really started)

5. Helper executable is configurable, allowing libcurl and aria2 backends (also not really started)

I'm still working on a few details (gdbus-codegen does not handle file descriptors well; you should interpret the annotations as returning a UnixFD), but the XML for the helper API is here:
https://github.com/Mathnerd314/ostree/blob/libsoup-2/fetch.xml

This approach should allow fully run-time-pluggable pulls, once finished.

Let me know if this could be done better in some way, since I'm in the process of implementing it. I thought about OstreeURI but it seemed weird to have a URI type for distinct concepts (e.g. the summary file and <some-hash>.filez); splitting it into different dbus methods seems like the right level of abstraction.

-- Allan Gardner