[Setup-tool-hackers] Please read and comment: Setup Tools whitepaper.

[Miguel de Icaza <miguel helixcode com>]

Hello guys,

   This is a white paper on the Helix Setup Tools, I am trying to keep
an accurrate representation of the vision of the Helix Setup Tools.

   This post serves two purposes: to notify a few people on what the
plans are for the Helix Setup Tools and what the vision is, and to
request comments and input from the actual hackers on what is
accurrate and what things are not.  

   For instance, I am not sure of what the status is for the work that
Bradford has done.  Ideally in Jannuary, when we move to the new
offices, we should try to get Hans, Arturo and Paolo over here (and
hopefully we can get Bradford at some point).

   Anyways, if you are on the CC list for this, it means that this
document is of relevance to you.  Please, send me comments and
suggestions before we publish this document.

Best wishes,
Miguel
		  
			The Helix Setup Tools
		  
	       Miguel de Icaza (miguel@helixcode.com).

		      (C) 2000 Helix Code, Inc.

* Abstract


	The Helix Setup Tools is a set of tools aimed at making system
	administration of desktop Unix systems simple.  Currently
	configuring a Unix system requires a lot of Unix knowledge,
	the Helix Setup Tools lower the barrier to setting up Unix
	systems, and address this in a consistent way across Unix systems

* Features of the Helix Setup Tools

	* Targeted end users.

	* Do not impose a new configuration database on the system.

	* Single user interface across different Linux distributions, Unix variants. 

	* Clear split between backend and front-end.

	* A consistent user interface for system administration across
	  platforms.

	* Provides clustering support to manage groups of computers.

	* Time travel.

	* Location management.

	* Provides desktop services through CORBA.

* History

	The GNOME project wants to provide a complete suite of
	applications, tools, components and libraries to allow free
	Unix systems to compete on the desktop space with systems like
	Windows and the MacOS.

	When the GNOME project started, we decided that we would not
	tackle the problem of writing system administration tools, as
	we expected the Unix vendor to know more about their system
	and expected them to provide the right set of tools for
	configuring and setting up their system.  So GNOME would
	provide the same desktop experience across operating systems
	and we would expect the vendor tools to be integrated into
	GNOME in some way or another. 

	As time passed a number of problems with the above approach
	arised, and they were not helping the GNOME project achieve
	its goal:

		* Lack of consistency across platforms: different Unix
		  strains shipped with a different set of tools to
		  perform system administration for no good reason.
		  Some people shipped COAS, other people shipped
		  LinuxConf, some people write their own system
		  administration tools.

		* Interface inconsistency: the various tools would be
		  written using different GUI toolkits which did not
		  help with consistency.  Some other tools would be
		  targeted to have multiple front-ends and would
		  choose a set of user interface elements that could
		  be implemented in both a graphical version, a text
		  version and maybe an HTML version.  

		  Although the idea seems sound, in practice the
		  result is a poor user interface that is not as rich
		  as it could as it targets the common platform,
		  rather than taking advantage of each presentation
		  system. 

		* Incomplete tools: the tools would not be complete,
		  so from time to time it would be necessary to open
		  up a Unix terminal and type in some commands as
		  root.

		* Tools would only run as root: some of the system
		  administration tools in some systems would only run
		  as root.  To a non-seasoned user, this would mean
		  logging out and re-logging in as root.  Eventually
		  the user would just choose to run always as root, as
		  he does not have to deal with this problem.

		* Poor human interfaces: the tools would have little
		  or no attention paid to the usability aspect.
		  Sometimes just a bunch of widgets put together, with
		  no way to help users achieve the task they are
		  trying to achieve. 

		* Application-based configuration, rather than system
		  configuration:  sometimes configuration tools for a
		  specific package would be written, with no attention
		  paid to the interaction of this component with the
		  rest of the system.  Why should a user need a tool
		  to export directories with NFS, another for
		  AppleTalk, and a third one for Samba?


* Helix Code's Setup Tools

	As part of Helix Code's strategy to improve the GNOME desktop
	we wanted to address the problem above, to drive GNOME into a
	position where it can be successfully used by non-Unix
	experts.  

	While we decided to start work on the Helix Setup Tools, we
	came up with a design that would satisfiy the needs of end
	users, and at the same time would be a good tool for seasoned
	system administrators to use. 

	Some of the constrains set on the Helix Setup Tools are:

		* To be a set of cross-platform tools, the same set of
		  tools should work across the various Unix strains
		  and have the same user interface.  It does not
		  matter if you are using Solaris, Slackware, Debian,
		  or Red Hat: the system administration tools will be
		  the same, and they will do the same things from the
		  user perspective. 

		* The tools will have to parse and analize the system
		  configuration as used in traditional Unix.  Then
		  when changes are made, the tools should do inline
		  editing of those files

		* There will not be a central database layered on top
		  of the Unix system.  The Unix configuration files
		  are the master files that will be queried and
		  manipulated. 

		* Information on the system can at any time be parsed
		  and analized to produce an XML-based reprensentation
		  of the system state. 

		* The functionality to parse and analize a system, as
		  well as the functionality required to make changes
		  to the system will be exported as well as CORBA
		  interfaces that will become system services.
	
	Given that the state of the system can be at any point
	can be snapshotted and turned into an XML description of the
	system, a number of possibilities are open by layering new
	tools on top of the core infrastructure, we will get back to
	this later. 

* The Helix Setup Tools structure. 

	Helix Setup Tools is a component-based architecture.  New
	components can be added to the system when new features are
	added to the system.  At the time of this writing the Helix
	Setup Tools have the following modules: file system, memory
	management, name resolution, network configuration, exported
	shares management, time and date and user management. 

	Each Setup Tool consists of two parts: a GNOME-based
	front-end, and a backend.   

	The backend is typically written in Perl, or C.  Each backend
	program has a number of well defined parts.  

		1. Operating System detection.  

		2. Parsing and analisys of the system configuration.

		3. Generation of an XML-based representation of the
		   state that was parsed and analized on step 2.

		4. In-place modification of the system files based on
		   a new configuration definition.

		5. Parsing of an XML-based definition of the system.

	That looks more complex than it actually is.  The bottom line
	is that every Setup Tool can be invoked with the --get flag
	and you would get an XML-based snapshot of the system for the
	domain that the Setup Tool administers.  Or you can feed a new
	system configuration by passing the --set argument to a Setup
	Tool.  

	For example:

		bash$ nameresolution-conf --get
		<?xml version='1.0' encoding='ISO-8859-1' standalone='yes'?>
		<!DOCTYPE nameresolution []>
		
		<nameresolution>
		
		  <!-- Configuration starts here -->
		
		  <hostname>erandi.helixcode.com</hostname>
		  <workgroup>MYGROUP</workgroup>
		  <description>Samba Server</description>
		
		  <searchdomain>helixcode.com</searchdomain>
		
		  <nameserver>140.239.238.2</nameserver>
		  <nameserver>132.248.10.2</nameserver>
		
		  <statichost>
		    <ip>127.0.0.1</ip>
		    <alias>erandi.helixcode.com</alias>
		    <alias>erandi</alias>
		    <alias>localhost.localdomain</alias>
		    <alias>localhost</alias>
		  </statichost>
		
		  <!-- You shouldn't have to modify anything below this line -->
		
		  <order><hosts/><bind/></order>
		  <hostmatch return='all'/>
		
		  <!-- You cannot modify anything below this line -->
		
		  <!-- End of configuration -->
		
		</nameresolution>
		bash$

	The backend and the front-end communicate by exchanging these
	XML-based representations.  

	The front-end would provide the best user interface possible
	to edit the above configuration, and then when the user is
	ready to accept the changes, invoke the backend with a new XML
	representation and pass the --set argument to it.

	The front-end does not need to know anything about the
	underlying OS, unless a feature would be present or hidden,
	but besides these things, the front-end is a generic tool: it
	does not matter what is running on the backend.

* The root-manager.

	The root-manager is a suid application that would authenticate
	a regular user to have administration privileges and run
	applications on his behalf as root.  

	The setup tools are integrated with the GNOME control center,
	and blend with the rest of the configuration tools for the
	user's desktop.  The first time the user tries to launch an
	application that requires root privileges, the root manager
	will be invoked, and will stay around to launch any other
	Setup Tools that the user activates without asking for the
	root password a second time.  

* The upper layers

	The fact that the Helix Setup Tools can create snapshots of
	the system at any point, and can change the system to match
	an XML-based definition of the system is the foundation for
	two extra features: Location manager and Time Travel.

	Neither of them have been finalized.  They are both works in
	progress, an we hope to release these tools integrated with
	the rest of the GNOME desktop in the near future.  Snapshots
	of these are available on demand (FIXME: we should put these
	tarballs in ftp.gnome.org and ftp.helixcode.com). 

  * Location management

	Location management is an application of the Helix Setup Tools
	that is of most interest to Laptop users.  For example, here
	is a typical setup, the one I am faced with:

		* Laptop at the Boston office: uses a static IP
		  address; local DNS server; has to use a Postscript
		  printer hooked up to a machine on the network;  Time
		  Zone set to EST;  News server is news.harvard.net.
		  Mounts NFS partitions from Helix network.

		* Laptop at Boston appartment: uses DHCP based IP
		  address; DNS server and news server are my providers
		  servers;  Printer is an HP 4 that requires
		  client-side Postscript to HP conversion, attached to
		  a Mac computer.

		* Laptop at Mexico appartment: uses PPP for internet
		  access; DNS server and news server are UNAM's
		  servers;  No printer defined;  Timezone set to
		  America/Mexico_City.

		* Laptop at Nuclear Sciences Research facility at
		  UNAM: DNS server is local DNS server, news server is
		  news.unam.mx;  Default printer is Postscript
		  attached to Linux machine;  Timezeon set to
		  America/Mexico_City.  

		* Traveling in the US: ISP-provided PPP;  No news
		  system; no default printer; Timezone changes;
		  Dynamic DNS setup. 

	The idea is to present the user with a panel applet, or
	integrate into the panel a button that allows users to switch
	their location configuration by just selecting a location from
	the list of locations.

	The location manager should allow the user to choose which
	settings change, and which settings do not change.  

	We want to store the differences between two configuration
	through a system like XPath (only the flat nodes are recorded
	for simplicity now, for non flat nodes, we will just consider
	its parent to be the last root node we can handle).

  * Time Travel	and Undo.

	Each time a change is made to the system configuration, a
	filter sits between the front-end and the back-end and
	archives all the changes.

	Consider the case where an end user has a working system
	configuration.  One day the power user niece comes home, plays
	with the computer, and figures he can `improve' the setup in
	the machine.  By the time the niece is gone, the machine is no
	longer working, and your mum is lost.  She has no idea what
	happened, but the machine has stopped working. 

	Time travel allows the user to restore the configuration to
	the state it was on a given date.  The archiving system can be
	asked to retrieve the configuration that was in effect at any
	time in the past since the installation of the Helix Setup
	Tools and restore that configuration. 

	The Undo system is layered on top of this, given that we have
	entire snapshots of the changes that were done to the system,
	we can move back and forth between the various changes made on
	the system. 

* Clustering

	We are looking at clustering from the point of view of
	managing a set of computers: performing the same kind of
	configuration and tuning that would be done in a
	machine-per-machine basis.

	Typically network administrators would need a set of changes
	to be applied to a set of computers in a network.  The Helix
	Setup Tools will allow you to create groups of computers and
	allow you to apply changes to sets of computers or sets of
	sets (for example: changing the DNS server for all computers
	in the company;  Or changing the print server for the
	engineering department;  Or changing the NFS server for both
	marketing and engineering;  Or changing the network
	configuration for the entire company to use DHCP).

	Clustering support is an ongoing development at Helix.  The
	current plan is to allow the user to define a "login" method
	in a per-computer, or in a per-group basis (OpenSSH-based
	login is our current approach).

	Then the central manager would be able to fetch the
	information for all the cluster, allow the administrator to
	make changes, and have those changes be applied to all the
	machines on the network.  This is done regardless of the kind
	of operating system being run, remember that the Helix Setup
	Tools do not care about the underlying operating system.  

	This code is still a work in progress.

* Implementation details.

	* Error handling, codes being passed back and forth.
	* XML based encapsulation of data.
	* The archiving setup.
	* Integration with the gnome control center.
	
* The current components
	
	FIXME: list the features in each one of these.

	memory:
	networking:
	time:
	disks:
	nameresolution:
	shares:
	users:

* The components planned:

	Mail configuration.
	Web server configuration.
	Printer configuration/Printer spooler.

* CORBA system services.

	The following text was taken from my `How to fix Unix' paper:

  * Background

	Configuring a Unix system traditionally has involved editing 
	a system file, typically on /etc and saving the file.  Then
	either the various programs and processes on the system pick
	up the changes, or you have to notify the daemons that require
	to be informed of this change.

	This is not really an ideal situation, because those files are
	easily corrupted by human error.  And depending on the tool,
	you need a different way of editing the file.

	For example, consider /etc/passwd, to edit this file you need
	to use the vipw command, which will do some basic checking on
	the integrity of the file before you save it (as well as
	locking), but there are no equivalent commands to "lock",
	edit, and verify other system files. 

	So depending on each tool, a user/administrator needs to:

		1. Read the man page, and figure out what the correct
		   procedure is for a specific piece to be edited.
		   This is obviously not always documented.  

		2. Edit the file.

		3. Now, depending on the kind of file you edited, you
		   need to figure out how to notify the proper service
		   that a change has happened.

		4. Depending on the service, either hope that the
		   settings are properly reloaded, or that some error
		   is going to creep in, or that something will be
		   said in some of the system logs.
	
		   For the system logs case, examine /etc/syslog.conf
		   to locate the place where error messages are
		   logged.  Then make sure you tail -f the file,
		   restart the service, scan for errors.

		   Hopefully you do not have a busy web server, or the
		   error is going to be lost in the noise (or
		   hopefully you edited /etc/syslog.conf, restarted
		   syslog)
	
  * What we want to do

	Although the above `Background' description was aimed towards
	moving applications into providing CORBA interfaces to their
	functionality, and making new applications have a CORBA-based
	control facility, this does not solve the problem of fixing
	the current Unix.

	Although existing tools could be modified, a stop gap measure
	would be to reuse the Helix Setup Tools.  Given that the Helix
	Setup Tools have already gone trough a great lenght to isolate
	the different across Unix systems, and given that they can
	analize, and modify existing system settings, we think that
	the backends are perfect for encapsulating the configuration
	issues of a Unix system.

	Each one of the Setup Tool backends will also be installed as
	a system service (only while running as root) that would be
	activated on demand to expose the system management functions
	trough CORBA interfaces.

	So the objective here is to add a few changes to the existing
	backends (this will be done by Helix CORBA::Perl team) to
	expose the CORBA interfaces for querying and making changes to
	the system, using the functions that are already defined in
	the code.  

	A programmer could write code like this in Perl:

	use Bonobo::bonobo;
	
	$interfaces = BONOBO::bonobo::resolveReference ("interfaces");
	$name_resolution = $interfaces.getObject ("IDL:Helix/NameResolution:1.0");
	print "Hostname is: ", $name_resolution->getHostname (), "\n";
	print "Changing hostname to JollyJumper\n";
	$name_resolution->setHostname ("JollyJumper");

* The Future

	It is possible to write other front-ends to the Helix Setup
	Tools.  A web-based front-end is possible, as well as a
	text-based front end.  We would like to see those happen at
	some point. 

	Helix Code has also been working on Red Carpet, a
	cross-platform package management tool, and update tool that
	will bring the same uniformity to all systems.  Red Carpet
	works currently on RPM-based systems and Debian-based
	systems.  We will be extending it to support the Solaris, IRIX
	and HP-UX native package formats as well in the future. 

	Package management is essential to maintain a system up to
	date, and to allow people to easily install software.  Red
	Carpet and the Helix Setup Tools will also be integrated to
	deliver a complete, perfectly blended set of services.

	The tricky part is using Red Carpet for doing cluster
	management. 

* The Hackers

	The Helix Setup Tools were implmented by Hans Peter Jansson,
	Arturo Espinosa, Bradford Hovinen and Tambet Ingo.  I am their
	project manager.

_______________________________________________
Setup-tool-hackers maillist  -  Setup-tool-hackers@helixcode.com
http://lists.helixcode.com/mailman/listinfo/setup-tool-hackers