Re: Relocatability of packages

[David Jeske <jeske home chat net>]

On Thu, Sep 24, 1998 at 09:22:28PM -0400, Tim Moore wrote:
> OK, that makes sense. But I don't like this hard separation between gui
> and non-gui. What if I want my package to have GUI *and* CLUI tools? Or if
> I want my one program to be usable in both modes? I don't think the GNOME
> project intends to deprecate the command-line as much as NS did, and I
> think that there's more of a portability concern, too. 

I can understand that, although I think it's misdirected, if GNOME
intends to start to use CORBA and really allow beginning end users to
use Gnome then apps need to provide everything via the GUI.

> I have strong reservations about changing around commonly accepted
> filesystem layout *by default* (though we should make it as easy as
> possible to do whatever you want to on your own system).

I can understand that as well...

> > works. Instead of the current Gnome/Linux fiasco where you have to
> > configure the window manager for the applications you have installed.
> 
> I don't know what you're talking about when you say you have to configure
> the WM for all the applications you have installed. *I* don't have to. Are
> you talking about the icon for minimized windows (which the app is
> supposed to provide via WM hints) or for the panel 

I'm talking about:
 - the icon for the app in app managers like WindowMaker's Doc
 - the icon for the filetypes an application can open
 - the icon for the application which will be presented when I click
   on this app in a file viewer.

> (which the apps install themselves in <prefix>share/apps)? If you're
> talking about some other launcher that your window manager has, then
> GNOME has already solved that problem.

Here is my scenerio: I'm an _end user_, I can't touch
"/usr/local/foobar/gnome/share/apps", that's where my administrator
installed Gnome.

> > System dependent data should be stored in libprefs....
> 
> I actually meant architecture dependent data. The nice thing about
> separating out bin and lib is that you can have one fileserver per
> architecture, and put all of the arch independent stuff (icons and
> whatever) onto a central server. (Unfortunately RPM doesn't seem to
> support this well anyway...I don't know about dpkg).

When it comes to architecture dependent data _nobody_ did it better
than Nextstep. As I explained in a previous email, you can _literally_
copy "OmniWeb.app" to a fileserver, and anyone from any platform can
just double click on it. They can login to the same account from
multiple platforms and it'll just work. They used FAT binaries to put
more than one architecture into the same executable file. I personally
would advocate using an "exports file" which would define where to
find the appropriate binaries for the apppriate platform with in
app-wrapper.

> What is libprefs? I assume its some library for storing preferences (duh)
> but is this something that exists for Unix? I'm aware of such a thing for
> BeOS, but I haven't seen it on Linux. Or are you just speaking
> hypothetically?

It was originally written for WindowMaker. It is available on Linux
today. The GnuStep people advocate it, and I thought many people on
Gnome were catching onto it also.

> And where would the libprefs data be stored?

It dosn't matter.. that's the beauty.  However, libprefs happens to
store it's data in some dot-directory off the user's home
directory. User specific preference data. However, apps never know
where it's stored, and thus can't depend on it's physical location,
they just depend on the libprefs api.

> > Furthermore, the unix /usr/local/bin path thing is only convinent if
> > your environment is sufficiently simple. Back at a former company I
> > went through the trouble to make a system which would custom build
> > paths based on a set of 'packages' which should be in the
> > environment. Essentially adding all the encapsulated wrappers to your
> > path for that given environment. It was the easiest way to (a) keep
> > environments repeatable as we moved forward, and (b) allow multiple
> > versions of the same apps to be installed at the same time. Both of
> > which are _necessary_ for any orginization which is more than just a
> > few people.
> 
> OK, I can't say that I've had to deal with an organization which is more
> than a few people. :-) I have a hard time swallowing that you need to keep
> lots of versions of the same apps around, but I'll take it on faith that
> you know what you're talking about. But it sounds like an administration
> and support nightmare to me. Why did you need to do that?

Because, in the real world you have to:
  - test software before you convert the entire orginization over to it
  - be able to recreate old environments to make sure you can produce old
    deliverables.
  - gradually move users to a new tool (while having both of them available)
  - allow different groups to use different tools without having to have
    completely separate fileservers, and install environments, for each
    group.

However, as simply an _end user_ I need to be able to install multiple
versions, because I will _always_ test a new version of a piece of
software before I delete the old one. I use Mutt for email, and I
always build mutt to go into an encap directory. I usually use the
latest and greatest beta, but if something goes wrong, I need to be
using a stable version _right then_, not two hours later when I've
downloaded, compiled, and installed the old version again.

> Well, frankly I don't like the throw-it-together-in-one-directory method
> for reasons relating to installation management, but more for usage. I
> think that packaging systems do fine for installation, and deal with the
> problems of throwing everything together pretty well. I honestly like RPM
> a lot better then Mac OS or BeOS where you could theoretically put
> everything in one directory but there were always exceptions like shared
> libraries which needed to go where the loader could find them, and
> manpages which have to go where man can find them (or the gnome-help
> browser) or command-line programs which have to go somewhere in my PATH...

I understand your dislike for these hybrid systems. Nextstep has much
better solutions. They have 'frameworks' for building and running
w/shlibs, and 'bundles' for gathering together shlibs and their
associated datafiles (i.e. for things like a gimp-plug-in). In fact,
they have it down to such a science that building is even made
easier. Havn't you ever been sick of making build lines which tell GCC
what the lib and include paths for everything are? In Nextstep you can
just say that you need "FoundationKit" and "MiscKit" and it'll just
add the appropriate paths to your build.


****

> > Why don't you enumerate the things which are important to you in app
> > installation and we'll put our lists together and see what systems can
> > meet the goals?
> 
> 1 users shouldn't have to do anything special (modifying PATH or whatever)
>   to use new packages
> 2 packages don't have to go in any particular place (NOTE: This may
>   conflict with the previous requirement, butdoesn't have to if the
>   sysadmin uses a sensible policy. Of course, the default should be
>   sensible, too)
> 3 handles dependencies gracefully
> 4 handles removals/upgrades gracefully
> 5 separates static data and dynamic data so that the static data can be
>   mounted from a fileserver
> 6 handles upgrades of dynamic data gracefully so that changes in the the
>   package don't clobber changes made locally (and preferably allows
>   merging the upgrades together perhaps by diff3-ing the original version,
>   the installed version, and the upgraded version)
> 7 separates platform-dependent and platform-independent data so that they
>   can be mounted from different fileservers
> 8 handles all packages in a uniform way, whether they are all-GUI,
>   all-CLUI, all-daemon, or some combination
> 
> If the sysadmin installs them in a reasonable location, RPM and dpkg both
> handle 1. It's unclear whether the .app system would -- this will require
> more discussion.

It depends on what you define as "use". I think of a system like Gnome
tayloring more to GUI launch of apps (like NS) than you do.

> The .app system handles 2. So does RPM, assuming that the package was
> built to be relocatable. Often they are not, but the .app system assumes
> that anyway. I don't know if dpkg allows that...it appears not. Note that
> this would allow normal users to install apps if satisfied.

However, in the RPM case, you break "1" if you mean "command line usage" above.

> dpkg does 3. RPM does too, but it's somewhat less graceful. .app would
> seem not to, at least as it's currently proposed.

.app handles dependencies very well, it's just at a different
granularity. wrappers can depend on other wrappers. You can't 'share'
files which are within a wrapper. However, most of the uses of this
kind of thing in UNIX that I've seen are more of a hack resulting from
standard UNIX installation methods being poor. For example, if
Netscape has a bunch of icons which come with it for its buttons,
nobody should be sharing them IMO.

> dpkg and RPM both handle 4 well. .app would handle it *pretty* well (you
> would just remove the old directory and optionally replace it with the new
> one) but misses some subtleties.

dpkg and RPMs do _not_ handle this well IMO... upgrading to me means
"leaving the old stuff alone and trying the new stuff" and dpkg and
RPM do not do this at all.

> dpkg and RPM both handle 5 well. I don't know about .app since that hasn't
> really been specified completely.

I don't know exactly what you mean by static vs. dynamic
data. EndUser/GUI apps shouldn't be storing mutable data anywhere in
the filesystem directly. They should use something like libprefs for
storing system/user preferences, and they should read/create datafiles
when necessary. There should be no other data.

> dpkg and RPM both handle 6 well, though neither of them do the diff3-ing,
> AFAIK. .app doesn't handle it at all, at least so far.

I don't think dpkg and RPM handle this well, because they don't handle
it. They don't impose any concept of 'mutable' vs 'non-mutable' data
on the filestructure. When you install balsa, which files are mutable
and which are not? With .app, you know exactly which files are
mutable, NONE of them.

> 7 depends pretty much on the packager with dpkg and RPM. .app explicitly
> *encapsulates* the two types of data.

You are definetly correct that with app-wrappers, it's very difficult
to mount platform specific files from one fileserver, and platform
independent files from another fileserver. However, it's much easier
to setup and manage these hetrogenous networks with app-wrappers,
because you can put the platform specific binaries inside the wrappers
and the app can just "work" from any platform.

> RPM and dpkg handle all packages the same. As it stands, .app only works
> for GUI apps (and only for packages which only have one executable).

Agreed.. but I'm also not saying that you can't wrap up a .app inside
an RPM. I'm trying to propose an imposed structure on Gnome friendly
GUI apps.

> > Here is my list:
> > 
> > - install multiple versions of the same apps
> 
> As I said, this confuses me a bit. It seems like it would be very
> difficult to keep the versions from conflicting. How, for example, do you
> handle configuration files?

If you are doing things right, you use something like libprefs. The
new version either chooses to read/write the same (per user)
configuration data, or it chooses to read the old and convert and
store the new data under a new key.

> > - allow user's to install applications (no superuser required)
> 
> We agree on this.

Good..:) Do you agree that this dosn't work currently with RPM, dpkg,
and Gnome's "share/..." concepts?

> > - trackability (i.e. the ability to delete everything related to an
> >   app/package easily)
> > - repeatability (i.e. the ability to recreate a requred environment/
> >   set of apps on another machine quickly and easily)
> 
> Both easy with package managers.

Yes, but my point is that apps arn't always installed with package
managers, and even if someone makes a package for your app, whose to
say it uses the _right_ package manager for your system.

Did you install _every_ thing which is in /usr/local/bin (or
analagous) on your system with RPMs?

> > - separation of 'immutable' (i.e. part of the distribution) app
> >   data from 'mutable' data. 
> 
> Agree on this, too. But you haven't really fully explained how .app
> encapsulation resolves this.

Data within the app is immutable. site/user specific configuration
data is stored via an API (like libprefs), data for a document is
stored in the document. That's it.

> Well, it can be tough to draw a distinction sometimes. GNU Midnight
> Commander is ostensibly a "end user" app, but comes with a server. On the
> flip side, what if I want to distribute my server with my nifty graphical
> configurator?

I agree.... so take all those apps which are hard to easily put into
one category or another and make them super confusing to install, and
I won't have any complaints. I'm advocating that for all the little
trinket GUI apps which people have written or will write (xanim,
balsa, wmprefs.app, xterm, xwinamp, etc) that they be app-wrappered.

> > The current Gnome system can not handle the first two. Because of
> > that, it can't handle "repeatability" either (because I can't recreate
> > an environment which needs Netscape 3 somewhere that already has
> > Netscape 4 installed).
> 
> But Netscape has other problems with conflicts that are internal. I can't
> switch back and forth between two versions because they have incompatible
> preference files. And even different users on the same system will run
> into problems with any program that uses system-wide configuration files.

I know, and that's why we should push Gnome apps to use something like
libprefs. I'm advocating that we create a more specific description of
what an app needs to be in order to be a Gnome app than just "it links
against the gnome libraries". Just like Nextstep did. For the record,
NS apps didn't have to be app-wrappered, people just grew to avoid
ones which were not, and the development tools made it so easy to do
it that pretty much everyone did.

> Still, I can see your point. Perhaps executables should mirror shared
> libraries somewhat. The Netscape 3 package could install:
> /usr/bin/netscape.3.0
> And the Netscape 4 package could install:
> /usr/bin/netscape.4.0
> You could specify to the package manager which version was the default,
> and it would make a symlink:
> /usr/bin/netscape -> netscape.4.0
> and associate that with the 4.0 package.

I'm an end user, I can't write into /usr/bin. :)

> But there are still problems. What about documentation? This will require
> some thought.

What do you mean "what about documentation?". It's real
simple. _EVERYTHING_ which ships with the app should be non-mutable
data which is installed inside the app wrapper. That includes the
binaries, that includes the documentation, that includes, the app
icon, that includes the information about what mimetypes the app can
handle, that includes icons for those filetypes, that includes a list
of what CORBA apis the app can understand for remote automation, that
includes _everything_. That's the beauty of the wrapper, it's "OO" for
package installation. 

The key point of wrappers is that instead of exporting these things to
the system because of where you install them (i.e. like the Gnome
concept of putting icons in <gnomedir>/share/...) You _passively_
export all that information from the wrapper. That way wrappers never
interfere with eachother, and their information is always published in
an easy to parse form.

> > I personally don't think it separates immutable
> > data enough. The "trackability" can be had right now only _IF_ you use
> > a package manager. So all the people currently doing "make install" on
> > source for Gnome apps don't get trackability.
> 
> I think it's reasonable to assume that if you want trackability, you will
> probably use a package manager. Even building out of CVS, it's possilbe to
> compile to an RPM and install that (that's what I do).

I'm really not even complaining about whether someone is going to use
RPM or not. I'm complaining that right now there is no way to install
a Gnome app as a user (i.e. non-administrator) and have it work, have
an icon registered, and have it's filetypes registered. That dosn't
have anything to do with RPM or dkpg, it's all Gnome.

-- 
David Jeske (N9LCA) + http://www.chat.net/~jeske/ + jeske@chat.net