Re: GnuCash page on GO site

[linas linas org (Linas Vepstas)]

On Mon, Mar 01, 2004 at 10:37:03AM +0100, Rodrigo Moya was heard to remark:
> >  For the gnucash engine, we implemented
> > 'qof' to solve that need.  Actually, qof is older than gnomedb, 
> > Derek's re-write of qof is now the 3rd rewrite of the query engine.  
> > I don't think libgda even *has* a query engine.  
> > 
> what do you mean by a 'query engine'?

To generate reports, we need the ability to perform queries
over objects.  For example, we need to find all objects of type
"split" between a start date and an end date, which belong to 
account XYZ;  this query is used to generate the checkbook register.

Another example: we need to find all objects of type "split",
for any date, that are not marked reconciled, and belong to 
account XYZ.  We use this to populate the reconcile window.

The query engine is *not* SQL, and it doesn't go to a database;
rather, it traverses a set of C language objects in system RAM. 
There is a hello-world example included with the source code
in http://qof.sourceforge.net/ Oddly, that example is not on
the website :-(

Note that we also have code that converts scheme queries to qof, 
and we also have a backend that converts qof to SQL. When we
run in SQL mode, only some of the objects may be 'realized' in
system memory; the rest stay unrealized in the sql db, thus we 
need to run the query there too.

-------
Independently, I have another abstraction layer called "dwi/dui"
which knows how to get and put values from various interfaces,
including sql, gtk widgets and glib gobjects.  I have been 
planning on stapling that to qof; this would allow me to make 
crazy queries like "find all gtk widget instances that are 
pushbuttons and are depressed".   


> > In my most recent adventure, I was intellectually contemplating
> > gluing the gnucash qof to the bottom of SQLLite.  sqllite does
> > support a limited amount of abstraction, I think they even have 
> > some sample code showing how to use sql to get the free disk space 
> > and cpu temperature.   I thought that would be pretty cool.
> > 
> > Derek, I'm not sure I ever mentioned/explained this: *if* we 
> > glue gnucash objects to the *bottom* of sqllite, then we can use
> > standard SQL queries to get data out of them.  Then people like
> > libgda and gnomedb and other report systems could sit on top of 
> > the sqllite api, and do whatever they need to do.  Yes, this is 
> > very hypothetical right now, and there are some dangerous 
> > unexplored issues.  It wasn't something I'd wanted to do soon,
> > just maybe someday.
> > 
> ugh, is that the abstraction you are looking for? I thought the cool
> thing was to hide SQL in the API, wasn't it?


Well, qof has some severe limitations: it doesn't have 'primary keys',
it doesn't do query optimization, the C API is clunky, it has no
scripting abilities, etc.  If we added these, we'd be re-inventing 
a rather sophisticated query engine.  So instead, it might be nice 
to use an existing engine.  The problem is that all existing engines 
(postgres, mysql) try to read data from a disk that is encoded in some
'proprietary' format.  I don't want the query engine to just read
static data off a disk, I want it to get dynamic data from me, 
i.e. to call methods on my objects; my objects then do calculations, 
and return data to the SQL engine, which then does whatever it does.
SQLLite can sort-of do this; it would still need some work.

Its not cool to hide SQL from the API.  Well, it is, but that's only
because SQL is broken.  Here's how I would like SQL to work:

-- the "power user" is allowed to write SQL queries, and associate
   them with reports.  This is in fact how many older SQL apps work.
   This power user has an incredible amount of flexibility in how
   they script things, and its not that hard to learn how to do this,
   its easier than being a programmer. 

   (Think of spread-sheets: "power users" can create spreadsheets 
   with rather complex expressions in thier cells, and there is 
   nothing wrong with that: the user can do very sophisticated things
   without having to be a programmer.)

-- The problem with the above is that the application doesn't know
   how to parse the SQL, and so doesn't know what the power user
   typed in.  (Only the database knows how to parse the SQL, and 
   there is no way for the application to ask what that was).  
   Since the SQL is opaque to the application, it can't do anything
   clever with it, it just has to be dumb about things.  This is
   the big reason why database apps are not desktop apps, and why
   desktop apps are almost never database apps. 

   For example, for GnuCash, I want the "power user" to write thier
   own custom SQL-like queries, such as "GET Balance WHERE Account=XYZ".
   The problem is that 'balance' is not a static bit of data, it
   must be dynamically computed.  Thus, my app (gnucash) needs to 
   be able to parse these SQL-like queries, examine them, possibly
   do something special (such as pop-up another GUI dialog), and
   then reassemble an new, different query to send to the database.

I beleive that if an application could "look at" an SQL query (examine
a parsed query), and react to that as it wished, then it would be
OK for the application to surface the query to the 'power user' .
Unfortunately, this is not possible today. (except, maybe by using
things like QOF or by some weird stunt with SQLLite).

--linas


-- 
pub  1024D/01045933 2001-02-01 Linas Vepstas (Labas!) <linas linas org>
PGP Key fingerprint = 8305 2521 6000 0B5E 8984  3F54 64A9 9A82 0104 5933