Re: [guppi-list] Guppi Update

[Jon Trowbridge <trow emccta com>]

On Thu, Jan 20, 2000 at 03:46:50PM -0500, Steven G. Johnson wrote:
> I built it on a Linux/PowerPC machine (Apple PowerBook G3 laptop), with no
> real problems

That is encouraging.  Thanks.

> It seemed to run fine, and I was able to successfully import a dataset and
> play a little with the Guile window (although I had to look in the source
> code to find what functions are available).  I'm quite impressed with your
> data importer; I've never seen anything so good at guessing file formats.

> Although I do like the data importer, I'm not quite as fond of the fact
> that I have to get through another couple of dialogs before the data is
> imported.  I would much rather have it just assume reasonable defaults, and
> then double-click on any dataset that I want to change the settings for.  I
> think, in most cases, the defaults will be acceptable, and one should
> optimize the interface for the common case, while still leaving the
> possibility of doing more complicated thing.

I agree that, since the importer is quite good at guessing defaults,
going through those dialogs is a bit of a pain.  I've got a few ideas
on how to deal with this.

* Magic codes in the data file itself.  If the importer saw a file
  that looks like this:

  ##!Guppi use-defaults
  ... data ...

  It would open the file w/ the defaults.  Something like

  ##!Guppi type=delimited delim=, cols=3 skip-until=67
  ... data ...

  could override the defaults.  Software that emits data files could
  be adjusted to simply prepend the appropriate header.  (Of course,
  the actual magic comment format would be something nicer than this.)

* Certain defaults could be bound to file extensions and/or
  directories in the .guppirc file.  So the user could specify, for
  example, that all .data files in /home/foo/data should be treated
  in a certain way.  (Or, more generally, files that match a given
  regexp should have a given set of defaults.)

* After a file has been opened once (by going through the dialogs),
  the parameters used to open it could be stored in a table somewhere.
  Subsequent visits to that file could then use the stored defaults.

* The data format guessing code could simply open the file using the
  default guess if it is extremely confident that it has guessed
  correctly.  Given the current state of the code, this would be easy
  to implement --- the key would be to think of a nice way to override
  this behavior: having two menu entries, "Open" and "Custom Open"
  seems clunky.

> Have you thought about a file format for storing the data in?  Are you
> planning on using a text-based file format (XML, perhaps?), or a binary
> format, or a both?

I'd been planning on XML.  Your point about binary formats for large
data sets is well taken, though.

Once the dust settles on my first alpha release, I'm going to move
data import/export into loadable modules --- so once that gets done,
you will be drafted to write an HDF5 plug-in. :-)

With module data importing, we'll be able to do all sorts of cool
stuff.  Data import from databases via sql queries, sucking in
information from lab data acquistion card and other real-time sources,
grabbing data from across the net via FTP/HTTP/whatever...

-JT

-- 
COMINT secret | Jam Echelon! http://www.echelon.wiretapped.net | NSA explosives
assassin terrorist car bomb tactical bin laden waco white house nuclear clinton