Re: Extensions Infrastructure Work

[Tim Cuthbertson <tim gfxmonk net>]

On Wed, Jun 22, 2011 at 12:24 PM, Jasper St. Pierre
<jstpierre mecheye net> wrote:
> On Tue, Jun 21, 2011 at 10:08 PM, Tim Cuthbertson <tim gfxmonk net> wrote:
>> On Wed, Jun 22, 2011 at 11:01 AM, Jasper St. Pierre
>> <jstpierre mecheye net> wrote:
>>>> Sorry if I'm late to the party with this suggestion.
>>>
>>> I'd rather late than never. I don't want to break this if people are
>>> unhappy with what I've come up with.
>>>
>>>> but I think this
>>>> is where using zero install[0] for extensions would shine:
>>>>
>>>>  - feeds that contain only JS (i.e no compilation required) are
>>>> trivial to create.
>>>>  - built-in GPG signing of feeds
>>>>  - version requirements are supported for dependencies (before, not-before)
>>>>  - you can depend on other extensions, or on arbitrary packages. These
>>>> can be other (vanilla) zero install feeds, or they can be feeds that
>>>> are a wrapper for a system package
>>>>  - system packages can have multiple names for a given zero install
>>>> feed. This will allow you to depend on fooBar for fedora, but
>>>> libfoo-bar on debian
>>>>  - 0install uses packagekit to resolve & install system packages, so
>>>> the experience should be fairly straightforward for users
>>>>  - doesn't require root access (unless system packages need
>>>> installing, obviously)
>>>
>>> I'm confused at what you want -- do you want 0install to be used for
>>> extension packages themselves, or for their dependencies?
>>
>> For extensions packages themselves. If dependencies are required, they
>> would have to be represented as a 0install feed (but this could just
>> be a reference to a package implementation if that is all that's
>> needed).
>>
>>> If you mean the former, I'll look at 0install if it does what I
>>> need.... and here's what I need.
>>>
>>>  * Users need to be able to click one button, and like magic the
>>> extension instantly is downloaded, unzipped, loaded and enabled. This
>>> is working correctly.
>>>  * Users need to be able to click the same button in the web UI, and
>>> the extension is disabled. This is, again, working correctly.
>>>  * I need to be able to inspect the state of the extension so I can
>>> enable the correct buttons. Right now this is done with the HTTP
>>> server.
>>>
>>>  * We need to be able to make this seamless for a simple extension
>>> case (no dependencies). I will not allow the word "0install" on a
>>> dialog when installing, enabling or disabling an extension.
>>
>> You could presumably use the 0install API (it's written in python) to
>> implement whatever UI you wish. In the typical case you would just
>> want to implement a download progress bar and a "do you trust this
>> person's GPG details" dialog for first-run of a new extension.
>
> If it's required that there's a progress bar, or "do you trust", I'd
> again like it to be in the web UI. This sounds possible, which is
> nice.

Sounds like it, yes. As long as the web UI can talk to the process
running on the client machine, that should be perfectly doable.

>> Note: 0install wouldn't change how extensions are enabled / disabled,
>> as that's pretty specific to GS itself. It would simply help with
>> distribution, integrity (GPG signed feeds), updates and dependency
>> resolution. All of which are pretty generic problems that shouldn't
>> need to be specific to GS.
>
> OK, this all sounds pretty excellent, but I'm not sure I'm ready to do
> that, at least for now. Everything that I listed above that I have to
> do when upgrading the shell doesn't change with 0install, so I'll just
> base it off the installation code I've already written. Tomorrow I'll
> remove the old mimetype handler approach from
> gnome-shell-extension-tool and stick the HTTP server into a new Python
> script so the patches are easier to digest. I'll download 0install and
> play with it a bit and experiment with exporting a feed from
> SweetTooth-WWW itself, along with the manifest.

Okay, sounds good. Let me know if I can help. The main point of
suggesting it now is so that we can use it when it makes sense to do
so instead of rolling new solutions to problems it already solves, but
it sounds like they're less immediate problems compared to enabling /
disabling extensions, etc.

>>> I'm editing the screencast that shows what I have worked out right
>>> now. Jason Clinton: what do you usually use for video editing, so I
>>> can stop cursing PiTiVi?
>>>
>>>> [0]: http://0install.net/
>>>>
>>>> I'm working up to releasing an extension that requires my own mutter
>>>> fork (for now, I hope that won't always be the case), and 0install is
>>>> the only way I can sanely do so. It allows my extension to
>>>>  - have dependencies
>>>>  - not mess with the system version of mutter
>>>>  - work cross-platform (not a big deal when fedora is the only distro
>>>> with GS, but will become more important later).
>>>
>>> How does 0install make the mutter case better?
>>
>> I'm not sure I follow. None of the above points can be done with the
>> current extension system as far as I know, so I'm not sure what I can
>> add. "the mutter case" is simply my reason for requiring bleeding-edge
>> dependencies and not conflicting with the system version of mutter.
>>
>
> OK, let's try that again. How does 0install sanely let users who
> install your extension use a bleeding-edge mutter?

In terms of what it actually does when you run my extension's 0install feed:

shellshape.xml defines an implementation (the extension files), a
dependency on the official `gnome-shell` (the system package), and a
dependency on my mutter fork (another 0install feed). So when 0install
"runs" my extension, it
 - ensures gnome-shell is installed (via packagekit)
 - downloads the appropriate binary for my mutter fork (a source
version is also available, but that's another story)
The my-mutter-fork.xml implementation is basically a tar archive, and
the feed (package file) specifies that when unpacked, the `lib/`
directory should be prepended to LD_LOAD_PATH. It also does a similar
thing with GI_TYPELIB_PATH and a few more. My extension's feed uses
the same mechanism to tell 0install to add its implementation
directory to XDG_DATA_DIRS so that gnome-shell can find it as normal
despite it not living in /usr/share/gnome-shell/extensions.

So when my extension is run, 0install will ensure an implementation of
my mutter fork is unpacked somewhere (in this case,
/home/tim/.cache/0install.net/implementations/sha1new=1e1762c4c78507c179cbf3105d61fbf585728e29).
It will set up LD_LOAD_PATH, GI_TYPELIB_PATH and XDG_DATA_DIRS, then
execute gnome-shell.

The sanity of such a solution is:
 - dependencies are downloaded as required
 - no global state. implementations live in a cache and can be
individually included as necessary by 0install, rather than being in
some system-wide (or user-wide) well-known directory that can only
have one version of a given extension.
    - This allows multiple implementations to be retained, which is
obviously useful for rollback.
 - dependencies are only active when they are required - i.e only the
things you depend upon are inserted into environment variables. If a
user decides to run vanilla gnome-shell after trying out my extension,
there is nothing to uninstall or revert to get rid of my extension or
my custom mutter fork - they're completely out of the way unless
explicitly included.

In terms of what this would mean for having all extensions distributed
via 0install, you would basically just need a list of feeds (these are
URLS) that the user has elected to use. You could attach preferred
versions to this data structure if needed.
Normally you would have these in a local feed file and let 0install
handle the launching, but for dynamic updating you can use the
0install API to figure out the appropriate environment variables for a
given set of feed URLs, and update the GS process' environment (from
within the process) accordingly on the fly.