Guppi status/plans

[Havoc Pennington <hp Mcs Net>]

Hi,

Several people have recently expressed interest in Guppi, so I thought I'd
write up a little text about the existing code and current plans for it,
for those who are interested. Basically this is a bit of propaganda.

First I'll talk about the goals of the project, and then a little bit
about the implementation so far. I am not a concise person so skim as you
see fit. :-) I hope you'll pass this message along to others who might be
interested in the project.

Here are the goals, or at least most of them:

  Data visualization (i.e. interactive, fast, on-screen display of data 
  in easily mutable formats; "playing" with data). 

  Attractive plots for printing, and for inclusion in presentations,
  etc.

  Statistics. Powerful statistical tools should be available in the 
  Guppi environment, so that you can transform and manipulate the 
  data, then view it, in a single integrated system.

  Easy-to-use. One role of the program will be as an embeddable
  component providing graphs in the Gnumeric spreadsheet. The 
  Gtk and Gnome libraries are being used. Arcane commands should 
  not be required.

  Interactive. Most plot programs construct a static description of the
  plot, then render it to postscript or to the screen. This is no good
  for data visualization. It is also no good for Excel-style plots, 
  where users can drag components of the plot around the screen, etc.

  Powerful. The Goose library combined with Guile scriptability allows
  you to perform fairly serious data analysis. As much as possible the GUI
  will expose this functionality, but power users will enjoy the
  interactive Guile shell. C gives us speed for crunching lots of 
  numbers, Guile gives a flexible interface to the C primitives.
  
  Extensible. The program should be written in a modular, reusable way
  so the C++ can be maintained and enhanced; Guile scriptability is
  provided for user-level customization. 

  General-purpose. Most programs now available were written by and for
  scientists. While Guppi will hopefully be useful in science, it should
  also be useful for education, finance, presentations, and tracking your
  personal budget. 

  Free. Of course. The license is GPL.

Implementation:

  First, Guppi does not start from scratch. A number of libraries are
  being used:

   - Goose provides statistics, and comes with Guile bindings. 
     Jon Trowbridge and Asger Alstrup Nielsen are the primary 
     authors of this software; Goose is a standalone library 
     useful in any statistical application and even runs on 
     Windows. 

   - Gtk/Gnome: These provide the tools to write a nice GUI, 
     complete with internationalization, etc.

   - Gnome Canvas: The canvas gives us a nice graphics engine 
     for the plot displays.

   - CORBA (via Gnome/ORBit): The main purpose of CORBA will be 
     to embed plots in Gnumeric. I am also using it for trivial 
     IPC to connect a Guile REPL to the Guppi process. Guppi 
     will probably be something of a testbed for the Gnome 
     embeddable objects framework (Bonobo).

   - Guile. Guile allows you to add scriptability to an application 
     in a matter of hours; it makes things embarassingly easy. It 
     also substantially simplifies prototyping and testing (you can 
     torture-test core functionality with Guile scripts, before even
     writing the GUI controls.)

   - XML: Daniel Veillard at the w3 consortium has donated a nice 
     XML library to Gnome, which is used to load and save in 
     Guppi. This means tricky-to-import data, or data you import
     then modify, is saved in a nice structured format. Also, plots
     can be saved. Generic XML tools can be used to manipulate 
     the resulting file.

   - libezpaint: This is a GPL library I am writing, basically a
     framework for manipulable GnomeCanvas objects (such as resizable
     squares, lines, circles, text). Basically equivalent to the 
     "drawing" facilities in Excel, PowerPoint, and so on.


  The application itself: things are very modular. First I have
  "libguppi"; libguppi contains basic objects, such as containers 
  full of data to be plotted. It contains the plot objects; these 
  are divided into "state" objects (ScatterPlotState) and "view" 
  objects (ScatterPlot). View objects render a representation of the
  plot's state to a GnomeCanvas. A state can have multiple views - this
  is important to work with Gnumeric, and also supports scripting well.

  libguppi also contains plot algorithms; such as "make up round 
  numbers to label this axis," "sort these two axes in parallel," 
  "divide this data into 'tiles' for rendering." 

  On top of libguppi is Guppi itself; Guppi contains Guile wrappers
  for much of libguppi, and provides a "shell" in which libguppi 
  does its rendering. One advantage of this design: we can eventually 
  write a relatively flyweight version of Guppi designed especially 
  to be embedded in other apps, sharing libguppi with the standalone
  Guppi but dispensing with some of the "chrome." A batch mode command
  line interface would also be possible.

  A major design goal is stability. In the Gtk tradition, Guppi is 
  positively paranoid about pointers; checking them at every turn, 
  setting them to NULL when freed, littering the code with 
  assertions. When possible C++ containers are used for safety.

  Another goal is code clarity and maintainability; there are many 
  existing plot apps with difficult to extend and hard to understand
  code. My hope is that nice C code plus Guile extensibility will 
  encourage hacking Guppi rather than starting anew whenever 
  someone requires different functionality.

  Target platforms: all systems that can run Gnome, with priority 
  given to free systems. Basically this means anything running X.

  There are a lot of interesting design problems; writing scalable
  plot algorithms that are fast enough for interactive use is 
  tricky. We want to be automagical - axes should automatically
  get labelled nicely, plots should be sized properly - but at the 
  same time, elements should be manually draggable. Everything has
  to permit multiple "views" and Guile bindings. 


  Status: 

  So far I've been working alone on the Guppi core, with Jon and Asger
  hacking Goose. Using the grep ';' method, there are about 4000 lines of
  'guppi2' code (12000 without the grep. :-). 'guppi2' is the CVS module
  I've created for my second attempt to write the software; attempt #1 had
  some bad mistakes, and so I've started again. 
 
  I may make an initial Guile-interface-only developer's release pretty
  soon; however, the code is always available from Gnome CVS. I expect to
  have a basic application with support for Goose statistics,
  data importing, load/save in XML format, and a couple types of plot 
  (likely scatter and timeseries) within a few months. 

  At some point I will have to pause and finish up libezpaint. I am 
  not sure how long this will take, and frankly I'm hoping someone 
  else will do it; if I have to stop and finish it Guppi development
  could stall for a while. The 'ezpaint' module in Gnome CVS 
  includes an extensive and detailed TODO list for potential hackers.

  I'm also hoping people will be interested in contributing different 
  kinds of plots they are interested in; bar plots, radar plots, 
  dot plots, 3D scatterplot, 3D bar plot, whatever people want.


Well, that's pretty much it. Questions and comments are welcome; 
participation in the project is more than welcome. Keep an eye on the web
site, http://www.gnome.org/guppi/, for the latest news. 

As you can see it's an ambitious project, but should fill a real void in
free software, and be genuinely useful to students, researchers, and
businesspeople around the world. 

Havoc