Re: GRAND MASTER PLAN

[Rodrigo Moya <rodrigo gnome-db org>]

On Mon, 2004-03-08 at 14:25 +1300, Dru wrote:

> >>Database low level abstraction.
> >>
> >>If we were to all use the same database abstraction layer then it will 
> >>make supporting a wide range of databases a lot easier. Also single 
> >>library to maintain and debug.  But everyone has their own code here.
> >>I've looked at both libdwi and libgda a lot but both still lack certain 
> >>minor features that are critical for my code base.  libgda has the low 
> >>level stuff but it also has support for all these other non-sql 
> >>databases and higher level stuff in there.  libdwi supports less 
> >>databases than libgda but is a pretty clean api.
> >>What if we ripped out the sql drivers,
> >>
> > 
> > which SQL drivers?
> 
> i'm talking about all the providers for sql databases.
> 
to rip them out? Why?

> >> put a new clean api on it and 
> >>told libgda to talk to that api instead of directly as it is now. Modify 
> >>all our projects to talk to this new library/project?
> >>
> > 
> > I don't get what you're trying to say here, which API would be used from
> > libgda? which SQL drivers would be ripped?
> 
> I'm thinking about the libgda providers for sql databases. At the moment 
> libgda has everything in it, i think it should be cut up into smaller 
> more managable peices.
> Heres an example directory layout as follows (assuming every directory 
> generated a single library):
> libgda_providers -> where you have all the providers.
> libgda_dbabs -> common database abstraction layer to talk to providers.
> libgda_tools -> All the useful tools you have for libgda and supporting 
> code.
> libgda_xml -> all the xml stuff
> libgda_meagrant -> the object abstraction layer. (GdaDataModel)
> libgda -> a library that contains all other libraries to maintain 
> backwarsd compitability and the overall api.
> 
that's a bit of a hell for maintainance, and for development, since you
need to know which parts you are using, which adds complication.

> My code could link to the libgda_dbabs and libgda_providers initially. 
> Reports would properly link to all of libgda.
> 
the report engine in libgda is in its own library. The XML queries stuff
could also be in a separate library, but the rest, it does not make
sense to me to split it into o many pieces.

> >>In regard to xml queries, a very useful feature but all my interfaces 
> >>are build around sql as is linas. I belive it would be more useful to 
> >>have a xml to sql and a sql to xml convertor or hide sql convertor away. 
> >>so sql'92 commands can be convereted into native sql commands.  And 
> >>proberly have that in the main libgda library.
> >>
> > 
> > there is a fully implemented XML query management thing in libgda. It
> > supports most SQL commands, and allows that SQL<->XML conversion. The
> > only problem with it is that we haven't used it much, but it should work
> > straight on.
> > 
> I think there is a lot of unforeen work in dealing with making different 
> sql dbs compitable with each other.
>
well, we planned to add the code to manage the XML query directly in the
providers, so that they can convert to their specific SQL dialect. This
is all ready in the API, only missing a full implementation in every
provider.

> > 
> >>Data abstraction layer.
> >>
> >>This layer that deals with dynamic data sets. Doing the write backs and 
> >>saves of data.  There is libgda, bonddb, gnueappserver, qof. Dont touch 
> >>gnueappserver, i wrote it and it suxs. I dont think we can look at 
> >>working together on this one until we sort out the low level abstraction 
> >>db.  How far are you down the track for qof linas?
> >>
> > 
> > bonddb uses native PostgreSQL libraries for psql access, IIRC. QOF, IIRC
> > what Linas said, uses a XML backend. So, and sorry for saying it myself,
> > the choice here seems to be libgda. I am talking just about the low
> > level data access stuff, not about the extra features bonddb and qof
> > provide.
> 
> bonddb use a bonddbwrapper library which uses postgrsql. I can replace 
> the bonddbwrapper library with libgda.
>
right

> Data abstraction layer i'm talking about is the bonddb DbObject, linas 
> qof stuff, and the GdaDataModel in libgda. Libgda aims to provide the 
> same functionality as bonddb so it makes it harder to intergrate unless 
> we refine the apis more.  Low level stuff should be in libgda yes. but 
> managing datasets is each has its own way of doing at the moment.
> 
hmm, what sense would libgda have if it just allowed to connect to
databases and not deal at all with the datasets?

I dont think, contrary to what Linas thinks, that libgda provides all
you need, but for the data access part, it provides all you need, IMHO.
bonddb has, on top of that (provided by libgda) a lot more features, so
I don't see the competition at all.

> 
> >>Reporting
> >>
> >>I think we definetly need a common ground here or not write our own.
> >>Unforuntly i already wrote my own, papyrus (stealing ideas from gnomedb) 
> >>when there was nothing providing reporting, now there are dozens of 
> >>reporting solutions out there (half are java/html though).  Linas, i 
> >>know how reports are done in gnucash and they are dynamic. A reporting 
> >>system should generate pdfs,ps,xml,html etc. And use the database low 
> >>level abstraction library. Can gnomedb people use papyrus or another 
> >>reporting engine, what is gnomedbs requirements?
> >>
> > 
> > gnome-db can use papyrus, yes, although for that, it would need to use
> > libgda, to make things easier to use / integrate.
> 
> I can switch papyrus other to libgda pretty easily. Only take a few hours.
>
great, would that make sense to you, after all these conversations?

> 
> > gnome-db's requirements are not clear, we wanted to provide a "powerful,
> > generic reporting system", with the low level stuff in libgda (look at
> > libgda/report for the code written so far) and a report viewer / editor
> > widget in libgnomedb.
> 
> What sort of low level stuff in libgda? 
>
opening databases, getting and modifying recordsets, tranactions, etc,
ie, the basic stuff you need when dealing with data sources

>   I havn't touched or done any 
> report viewer or designer. I used gnome pdf viewer for my report viewer.
>
well, that might be an easier solution to the problem. We can just
generate the PDF from a GUI and use the PDF Bonobo component to display
the generated PDF.

> Should papyrus be add as a seperate compoonent to the 
> libgda/meargent/gnomedb set?
>
if you want to, yes. The original idea in libgda was to provide, along
with the reporting feature, an API for apps to eaily manage reports
themselves. Could papyrus provide that? If so, I guess it could be into
libgda, but as a separated package, if needed.


> > Also, as some people have mentioned, it would make it really easier to
> > have all that data available to Gnumeric, Abiword, etc, for reporting,
> > mail merging, etc, etc
> 
> Use libgda with there gdadatamodel objects to make it avialible to 
> abiword/gnumeric. I think this should be seperate to reporting.
>
whatever, I mean the data you use for the report is available from any
app that uses libgda.

cheers