Re: Red Hat packaging (and hello all)

[David Elliott <dfe infinite-internet net>]

Gregory Leblanc wrote:

> On 29 Apr 2001 16:28:39 -0500, David Elliott wrote:
> > Hello,
>
> Howdy!
>
> > It's sad to say it, but RPM documentation is extremely poor.  Over the years
> > I have picked up several things.  There needs to be a set of guidelines for
> > gnome packages to ensure quality control and ease of installation.
>
> This is right at the top of the list of things to do on the GPP projects
> page.  :-)
>

:-)

>
> > While everyone is focusing on binary packages, source packages are every bit
> > as important.  I'll admit I am not very familiar with dpkg, but here are
> > some things that definitely apply to RPM and might apply to DPKG.
> >
> > 1. The packages should always be built as a non-root user.
>
> Absolutely.  This means that you don't get stray files lying all over
> your build system, and that building packages can't clobber anything
> that you already have installed.  Any other reasons for this?

I'd say mostly the stray files, that one bit me in the ass several times back in
the day before I learned about BuildRoots.  Then again, I was going on the then
current RPM docs which didn't mention them at all (or maybe mentioned them in the
very last chapter and then not for very long).

>

>
> > 2. The source packages should be rebuildable with a simple rpm --rebuild
> > package.src.rpm
>
> We should also make this work from the source tarballs that are the way
> that GNOME is really distributed.  Shouldn't be too hard, I've already
> talked to quite a few hackers, and most have no objection to somebody
> actually committing good spec files.
>

Unforunately, that is not very fair.  Do you want to have the equivilant for
every package system under the sun in the sourcetree?  Probably not.  On the
other hand, I believe if people want to actively maintain it then it has every
right to be in the source tree.

One advantage to RPM .specs is that a simple reading of it can provide insight on
how to compile manually or a start on how to build a package for a different
system.

>
> > 3. Packages must have appropriate Requires and even more important have
> > appropriate Prereqs
>
> Even more important than this is BuildRequires.  These are the packages
> that are required to be installed when you issue an rpm command that
> builds the binary packages from source.
>

Ah yes.  I was going to mention that but for some reason was thinking that there
was no BuildRequires.  Probably because almost no package uses it.

>
> > 4. Autoconf is great and all, but if a certain library is not available then
> > package building should fail.  I cannot think of any specifics at the
> > moment, but if a package is missing functionality because the system it was
> > built on didn't have libfoo.so that is not a good thing.  Part of this
> > involves maintaining configure.in and friends appropriately.
>
> Right, BuildRequires is the correct solution to this.  I'm assuming that
> dpkg has something similar, but I'll let the people who already know
> debian worry about that.
>

:-)

>
> > 5. RPM supports relocating packages, it would be good to support this.  For
> > example, it may be desirable to install a newer version of gnome packages
> > into /opt/gnome to test it before going ahead and putting it in the standard
> > /usr heirarchy.
>
> Does anybody other than Jeff Johnson actually understand how this works?
> I've read the stuff in Max RPM, and asked on the rpm list a couple of
> times, but I still don't get it.  Aren't there some GNOME packages that
> have paths hard-coded, and can't be moved?
>

Yeah, that is the downside.  In order for relocation to work right you most
definitely can't use rpaths nor can you hard-code the path in a way that is not
changable from a simple %post script.

I believe when the %post script runs you can determine what got relocated and
take appropriate actions (I think).

>
> > 6. RPM scriptlets are just that, scriptLETS.  That is mini scripts.  The
> > scriptlets should do the bare minimum for making the package run.  They
> > should not reconfigure half of the system.  There is a huge difference in
> > what is considered right and proper for a debian package to do and what is
> > considered right for an RPM to do.  Debian has interactive configuration.
> > RPM does not.  There should be absolutely no scriptlets that requires user
> > input.  There should also not be any scriptlets that require user
> > interaction after installation for the package to work correctly.
>
> Right, we should try to stick with the established guidelines for
> creating packages, no matter what our personal philosophy may be.  RPMs
> are -not- installed interactively, that's all there is to it.  I don't
> think that GNOME requires very many scripts to be run at all, outside of
> 'ldconfig', so this shouldn't be a big problem.
>

w00t!

>
> > 7. Try to maintain the users configuration as much as possible.  Red Hat's
> > apache packages are bad about this (although it has a lot to do with the way
> > apache's config file is one large file).
>
> I don't know too much about this one offhand, but I seem to recall some
> way of setting the behaviour for RPM so that files don't get clobbered
> on install.  I just can't remember what it was.
>

You can set noreplace in the .spec for the package, but every so often RH decides
to change something big and thus reverts you to the default files.

>
> > There are many more guidelines I am sure, but those are off the top of my
> > head.  Most of these are actually my pet peeves with RPM packagers.
>
> Well, we'll try not to peeve people with our packages.  :-)
>
> > On a final note:  I love writing specs and working out the build process and
> > designing really good packages, /BUT/ A 350 Mhz Pentium II is a bit slow at
> > compiling gnome.  Given that as previously mentioned an 800 Mhz machine
> > takes about a day to build gnome I would guess it would take my machine 2
> > days.
>
> Yeah, this sucks.  I've been thinking about how to solve this one a bit,
> but I don't have a great solution, yet.  For our initial work, which
> will be scripts or hand packaging, this machine just needs a metric
> boatload of ram and disk, and a fast processor.  What I'd like to do is
> set things up so that if we each have accounts on this machine, we're
> not wasting too much disk space with 30 versions of the same source or
> binary packages.  I just haven't figured out how to do that yet.  :-)
>

The problem is also how do we avoid clobbering over each other.

I suppose you could set up some space and chmod g+ws the directory and assign it
to a group that everyone would belong to.  You could even have several groups for
different packages.


>
> > Also, when building packages it often happens that you will have included an
> > extra file in the files list or something stupid like that causing the past
> > 2 hours of compiling to be lost because RPM will not package something that
> > it did not build itself.
> >
> > To deal with this situation I generally do something like this in the
> > package:
> >
> > %if "%{fullbuild}" == "0"
> > %{warn:YOU ARE A BASTARD. DON'T RELEASE THIS }
> > %else
> > %define fullbuild 1
> > %endif
>
> I don't have any idea what you're trying to do here, can you explain a
> bit more?
>

Hmm, okay, description down below.

>
> > Okay, so maybe the warning is a bit harsh.  Fortunately after building one
> > of these assuming you used -ba then you can do a --rebuild on the source
> > RPM and thus you have killed two birds with one stone (1. You know that
> > --rebuild works, and 2. You know that the package actually builds
> > correctly).
> >
> > Then you do something like this:
> >
> > %prep
> > %if ! %{fullbuild}
> > %setup -D -T -n linux
> > exit 0
> > %endif
> > %setup -q -n linux
> >
> > The last line there is the usual %setup macro, the stuff in the %if block
> > simply makes sure you are in the build directory for each section of the
> > build process.
> >
> > For the build section you want something like:
> > %build
> > %if ! %{fullbuild}
> > exit 0
> > %endif
> >
> > Assuming you did something like:
> > rpm -ba kernel.spec
> > And then found out during the build that you needed to patch the source or
> > you were missing files or something then you can do:
> > rpm --define "fullbuild 0" -ba kernel.spec
> > That is assuming you were already done building the whole thing and you just
> > want to install and package it now.
>
> I'm still not clear on what this accomplishes.  It looks to me like you
> make it quit if you're not running a "full build".  I can't figure out
> why you'd want to do that.
>

Alright.  Let's say you have got your specfile worked out and you start building
your package and it unpacks and compiles (the prep and build phases).  That is
actually usually the easy part of bulding an RPM.  It's fairly straightforward,
unpack the sources configure and make.

What's a PITA is figuring out the install process.  Often times you will have a
rouge makefile that doesn't install into the buildroot but tries installing in
the configured prefix (that is one example).  Other times you simply don't really
know what files are actually going to be installed and you need to see them in
the bulidroot to determine what to do with them and how you should divide them up
into different subpackages.

RPM does provide the so-called "list check" for this.  However, every so often
I'll have some stupid typo or error in the spec which won't show up until the
packages are actually built.  Or sometimes I'll run out of disk-space and have to
go on a cleaning binge.

So, you are in a way exactly right.  I exit 0 from the %prep and %build sections
if not doing a full build.  This is assuming you already have done the prep and
bulld phases and the source is compiled and ready to be %installed and packaged
up.  Since the prep and build are bash scripts running as children of RPM, the
exit 0 makes them exit successfully and RPM is none the wiser.  Then RPM will
execute the install phase and start packaging everything up.

Normally you would have to go ahead and rebuild the whole damn thing, only to
find out there is some other damn error preventing RPM from actually doing the
packaging.  So then you rebuild again.

With this system in place I generally write the prep, build, and a skeleton
install section and then go back and write a correct install and files.  Then
I take the SRPM and --rebuild it.  It's saved me countless hours, especially
given the long build times on my machine.  The kernel as I configure it for my
packages takes around an hour to build IIRC.

Come to think of it, I'll bet the real reason that prompted me to do this was
constantly running out of disk space when building kernels.  Since the build
process has to copy the source tree over into /usr/src/linux-* and then sed the
correct path into the dependency info and then make clean it takes a good 300 MB
or so.

>
> > Anyway, I am really looking forward to joining the packaging team!
>
> Welcome!  We're certainly glad for more volunteers!
>     Greg
>     GPP Founder and Leader

Well, I am certainly glad that someone finally made an official packaging
project.  From the looks of the mailing list so far, it seems that there is a lot
of talent and this project will get going quickly.

-Dave