Re: GRAND MASTER PLAN

[linas linas org (Linas Vepstas)]

On Mon, Mar 08, 2004 at 01:43:44AM +1300, Dru was heard to remark:
> 
> Overall vision:
> 
> A complete database toolkit set that allow the easy rapid application 
> development of custom db appliations for gnome. Also a collection of 
> common business applications like accounting, payroll, inventory etc.

Yes. Although I already have a problem with terminology.  
"database" is the wrong word, as it implies sql. Its broader
than that.  I want two abilities:

1) the ability to take data, be it from sql, an xml file, or 
   something ephemeral that came over a socket, and convert
   it into a set of C-language objects that can be manipulated
   and searched and operated on.  (Note that a possible data
   source is a gtkentry widget...)

2) the ability to suck data out of the c-language objects, and
   shove that data back to an sql database, or to an xml file
   or to a socket, or into a gtk widget (for display) or into
   an abiword document (for display). 

The reason I like C objects is that as a programmer, I can 
do all the whiz-bang application-specific coding easily when
I have plain-old c objects to work with.  As a lazy application
programmer, its a real hassle to get the data into and out of
xml, sql, etc. These data formats are "inaccessible".  So my 
vision for the toolkit is something that converts these 
"inaccessible" data sources/sinks into plain-old c objects 
that I can easily work with.

> Forms creation.
> I think we all seem pretty confident in glade here. thats solved.

Uhh, forms and reports are two sides of the same coin.  Remember
my 'bugzilla' analogy:  The bugzilla "report" for bug 123 is also
a "form" that allows the user to modify bug 123.   I'd like to 
be able to embed widgets into abiword docs, and use libglade-like 
api's to find & work with those widgets.

> Database low level abstraction.

When you say "database" here, you mean "SQL".  At the low-level, 
I find an api similar to libpg (the postgres api) entirely adequate.

But for other data sources, such as a glade-designed gtk "form",
it is a mistake to try to pull data out of that form using a 
libpg-like api.  Similarly, I don't really beleive that a libpg-like
interface is the correct api for an LDAP database.  Ditto for XML,
which is where my vision and the libgda vision differ significantly.
So, for low level abstraction, I beleive its best to stick to special 
purpose api's designed for that technology.  

The higher-level abstraction layers is where we need to make 
sql, xml, ldap, file and gtk/glade  form data look 'source 
independent'.

> Data abstraction layer.
> 
> This layer that deals with dynamic data sets. Doing the write backs and 
> saves of data.  There is libgda, bonddb, gnueappserver, qof. Dont touch 
> gnueappserver, i wrote it and it suxs. I dont think we can look at 
> working together on this one until we sort out the low level abstraction 
> db.  How far are you down the track for qof linas?

Not that far.  I really need to prototype the conversion of my 
'generic C objects' to/from SQL in order to confidently say 
"this approach works".  I was going to do that last christmas...
I have all the pieces ready and checked into cvs,  I have the 
master plan in my head, but I have to assemble the damn thing. 

If I had this prototype, then I would be mentally prepared to 
talk about things like 'triggers' which I know that you use. 
Mentally, I'm not there yet, not enabled for input.

> Sql parser.
> 
> I'd suggest moving libsql into a seperate library/package to make 
> maintaining it easier or moving it directly into a sql low level 
> abstraction library.  

Most important comment: the sql parse doesn't belong in the 
'low-level' abstraction library.  It goes higher up.  I think
virtually all of my flaming is about the confusion between what's
low level and what's high-level.  The low-level sql abstraction
library needs to be able to send ascii sql strings to the
sql database server, and present the row data returned from the 
sql server.  The low-level library should do nothing more than 
this.

Next: waht you call the "sql parser" is really two things: 
-- some code that turns an ascii sql string into a query,
   (I'd call this the "parser")
-- some code that performs the query on a set of data.
   (I'd call this the "query engine")

The query part of QOF, (written by Derek Atkins), is just the
second part.  I would be willing to abandon that code in 
exchange for something that both the parsing and the querying.

However, Derek will have my head on a plate if this other thing
isn't as easy to use as QOF.  Recall, the query engine in QOF
runs its query over generic C objects.  This ability cannot be
abandoned.  I can't replace this by some weird hack, it has to
be replaced by something legit, something that was designed to
run generic queries over generic objects. 


> Reporting
> 
> reporting engine, what is gnomedbs requirements? Linas, maybe a gtk 
> output engine or a xml to gladexml converter, or use the existing html 

Reporting is on the same coin as forms, its just on the other side.
A "form" flows data from somewhere else to my application. 
A "report" flows data from my application to somewhere else.

This is THE "high level" abstraction layer/API, to me. 

My app codes to this high-level API.  It then picks a particular
driver.  There is one driver that will push data into an abiword
doc.  There is another driver that will push data into a pane
of gtk widgets. There is a third driver that will push data 
into xml.  There is a fourth driver that will pushd data into
sql. A fifth driver will push data into ldap.  A sixth driver
builds a web page and pushes it out to Apache. Etc. 

(My "DWI" project was my attempt to create this high-level API.
I currently have drivers for glade/gtk and sql.)

Its possible that gnomedb and/or libmergeant has this kind of 
ability, but if so, I haven't figured it out yet.  Its not
clear how to create new drivers.

> Report Designer.
> 
> None of have anything here (expect gnue).  Its needed. I was thinking 
> orginally using openoffice or abiword to design reports but i really 
> havn't started looking at this at any detail. Any suggestions on how we 
> are going to achive this? Does this sound feasible?

I've been trying to explain to Martin Sevior what my vision for abi
is, and I think he almost gets it. I thnk once he gets it, it will be
easy.  I want abi to have a libglade-like interface.  The report
designer creates a named text block, and my program can then change
the text/color/font of the named text block.  I can't beleive this
is going to be hard.

--- 
Note that the above should be called the "abiword report designer".
The "gtk report designer" is still glade.  The "xml report designer"
is presumably some hand-written DTD's or xml schemas or something.
Some types of reports don't need report designers.  

The point again: the "report" is really just a pushing out of data
from my app to some other place.

> Finincial Objects.

Excellent point.  I have my heart set on "workflow objects", which 
deal with users and seeing which users can make modifications to 
which objects at what point in time.  e.g. only a person of type 
"manager" can approve of "chair purchase".  This might sound
wildly off-topic, but its not:  The point is that these "transactions" 
should be easy to build on top of the data abstraction layer.

If it is hard to build these types of objects on the data abstraction
layer, then we will have failed with the data abstraction layer.

--linas

-- 
pub  1024D/01045933 2001-02-01 Linas Vepstas (Labas!) <linas linas org>
PGP Key fingerprint = 8305 2521 6000 0B5E 8984  3F54 64A9 9A82 0104 5933