egcs and AsciiImport

[Jon Trowbridge <trow emccta com>]

I was playing around a little bit with Asger's AsciiImport code, which
has the unfortunate habit of putting egcs into some sort of a
memory-slurping loop.

I found that I could get test_ascii_import to compile by rearranging
the code from this style: 

template<class X, class Y>
class Foo {
public:

void bar() {
  ... lots of code ...
}

void car() {
  ... lots of code ...
}

... many definitions omitted ...

};

to something like this:

template<class X, class Y>
class Foo {
public:

  void bar();
  void car();
  
... many definitions omitted ...

};

template<class X, class Y>
void Foo<X,Y>::bar()
{
  ... lots of code ...
}

template<class X, class Y>
void Foo<X,Y>::car()
{
  ... lots of code ...
}

... many functions omitted ...

It still takes forever to compile (40 seconds on a P2-300 w/ 128M),
and is a huge memory hog (grows to 20M fairly quickly, then jumps up
to 42M for a split second, then drops back down and hovers around
25M for the rest of the build), but it does finally build.

Of course, this still sucks.

Now the best solution for all of this is for egcs to stop being
broken.  But assuming that doesn't happen anytime soon, maybe there is
something else we can do.

While poking around in the AsciiImport class, I noticed that (as far
as I can tell) most of the code does not depend on the template-ness
of the class.  The template args specify the input type (and the gui
output type... but I'll ignore that for now.  Everything I say about
input should carry over to output), but all of the functions (as far
as I can tell) just get their data via the readline() function.

So rather than have all of that code in a massive header, why not:

* Create a new non-template class, say AsciiImport_Base, that contains
  all of the code that is now in AsciiImport but defined readline() to
  be a pure virtual function.

* Then create a new AsciiImport<Input,Gui> template that derives from
  AsciiImport_Base and defines readline() appropriately.

* It might also be useful to create other specializations, like
  "AsciiStreamImport", that aren't templates at all but are optimized
  for input via iostreams.

* And if anyone is feeling very ambitious, why not create an
  AsciiFileImport which can mmap() a file to pull in data extra-fast?
  (Of course, this might not be portable to win32, but some #ifdef
  magic could take care of that.  And maybe avoiding a few buffer
  copies in the kernel wouldn't really save us that much...)

Now there may be some good reason why this won't work... but if we
could do something like this to avoid having to deal with huge
templates (which always seem to be problematic <grumble>), I think
that it might be worth it.