Re: [gnome-db] Papyrus, xml reporting

[Andrew Hill <andru treshna com>]

On Wed, 2001-12-26 at 01:46, Rodrigo Moya wrote:
> On Tue, 2001-12-25 at 11:20, Andrew Hill wrote:
> > > > Ok i've started coding but not much coding.  As in only got a small
> > > > handful of C files and not even in CVS.  Been working on database
> > stuff
> > > > with nasty things like sql parses.  We are going for just postgresql
> > > > direct support initially, just for ease of testing, and then we will
> > put
> > > > gnome-db and bonddb support in straight after we got layout sussed.
> > > > anyway. xml file. I've attached it and it gives a good idea of what
> > the
> > > > end thing will look like.  Its slightly different to your xml.  
> > > > 
> > > 
> > > I have seen the .xml file and I want to ask you if could you send us
> > > patches for our DTD as modifications request to
> > gnome-db-list gnome org,
> > > I mean, with a few lines with a description of changes done.
> > > 
> > > Also, I want start to implement all this in HEAD branch (GNOME 2.0
> > > version) so I need to work with you (and you with me) to do all as a
> > > team work.
> > 
> > Anyway my team (myself, andre (lead), and liam as at the moment) are
> > working on reporting system in our spare time.  We labeled it papyrus.
> > If it should be called something else and you have a better suggestion
> > let me know.  We want this reporting system to work with gnome-db though
> > still have some backend independent.  
> > 
> perfect! From what I've seen in the code, I've had some ideas about the
> backend independence. That is, in the XML file:
> 
> <group datasrc="gda:select * from foobar">

You would specify bindings libary in the report header, it could default
to gda but you could also specify postgresql.  then each datasrc would
be ="select * from foobar"

Each database libary, ie gda, postgresql, bonddb, mysql etc is done is
handled by the bindings and you just link at runtime.  At the moment
only a dummy one is written, i'll write a postgresql one at some point.
maybe even today. and a gda one when i get the rendering finished unless
someone else does it.

> this will, obviously use the gda library to get the data. Then, we can
> add more datasource kinds (postgres:, xml:, whatever:), and for this to
> work great, we can have a plugin system (via CORBA interfaces) so that
> you can easily add datasources to the report engine, having a set of
> default ones.
> Then, to get the data, we'll just use a GdaDataModel, which is an
> abstract/database independent array of data. So, the report engine, when
> getting its data, will parse the "datasrc" property from the group node,
> and depending on the prefix (gda:, postgres:, xml:, etc), it will get
> the data from one or the other datasource. That's the good thing of the
> GdaDataModel's, that we can easily add new data sources to libgda.
> A nice default datasource could be a xml: one, so that you can get data
> from XML files generated by the gda-export utility, and thus run reports
> without no database connections at all.
> 
> > gnome-print is used a lot for the actual printing, previewing etc.
> > though we ran into a few minor short comings with it which wont take
> > long to modify gnome-print to resolve these (like not been able to run
> > reports non-interactively).
> >
> hm,, gnome-print should not be used (at least libgnomeprintui,
> libgnomeprint would be ok) in the report engine, since this will make it
> depend on GTK, which is not good.
> The preview stuff should belong to the UI parts of the architecture
> (libgnomedb)

libgnomeprint still depends on all the gnome stuff.  I know its bad been
tied to UI libary like gnome-print but we looked at 4 different libaries
and gnome-print was the most commonly installed feature rich one of them
all and didn't have any annoying little bugs that screwed up stuff in
the first prototype.  you could take the render_print.c and print.c into
a different libary, though there are parts of the xml parser that
currently use gnome, in regard to fonts etc.

 
> > libxml is used for rendering xml, langauges can be embedded into the xml
> > but we are still fussing over python (has advantages for linking data
> > but disadvanages with its tabbing), slang (its a bit messy) and guile
> > (well its guile)..  
> >
> that is a very nice idea. We could just even support, later on, several
> languages, by just using something like:
> 
> <language type="perl">
> print "Perl";
> ...
> </language>

yip be good to support many langagues ultimatly, though someone else
will have to code it.  

> > postitioning is both relative and absolete.  relative been the first
> > choice.  data is managed in groups, and groups can be sub sets of
> > groups.  Report generation isn't single parse and the whole thing is
> > generated in a 2d workspace before been dumped to gnome-print. This
> > makes it resource hungry but it also gives us the best felixablity.
> >
> my idea, on the report engine part, is to just return the XML filled in
> with data from the different data sources. Then, the rendering
> (conversion to gnome-print for previewing and printing) should be done
> on the client side, so I think it is not a big problem if it is resource
> hungry, as this will be done on the client side.

arr ok now we differ.  I was using the XML to define the report, similar
to how you would use HTML to define a web page. And then report server
would read the xml and generate a report (ie a pdf file).  though the
report system could only be client side effecivly.  The xml was not
going to have invidiual row data in it but this would be handled in
memory with structures.  though this is one differing approach to xml.
i'll ask andre about it.  
  
> > I'm not worrying about the details of the xml files or the database
> > intergration at moment as this is easy stuff.  At the moment i'm
> > concentrating on getting the xml overall structure right and making sure
> > we can correctly represent 99% of the reports we would write with it.
> > The embedded langauge wont at the moment let you reorganise the way
> > layout is provided, most of the time you wont need this as your using
> > the langauge to work out what values are displayed where like calc
> > totals.  
> > 
> yes, datasource stuff is very easy to add, as I said, even supporting
> multiple datasources.
> 
> > what we arn't working on but would be good if we could have is a front
> > end for generating reports.
> > 
> Jimspbox aol com is interested in working on this part, which, as I
> said, should be the one using the GnomePrint preview stuff, as this is
> the real UI thing.
> 
> > In regard to your DTD's your definations of xml tags will remain very
> > similar, but the structure is different with the use of groups, group
> > headers etc.  I prefer to leave DTD's till i get the thing generating a
> > sample report, which it doesn't do at the moment. the state of the code
> > is that it just parses the xml and greats the nessary layouts and prints
> > lots of debugging info out.  it'll be another week or 2 at least before
> > we have it generating simple reports.  though i have a forced deadline
> > on this one for mid jan. aggrr.
> > 
> > regarding cvs.
> > i'm running this from my cvs server at the moment but i'm fine with
> > moving it tsover to the gnome cvs server but i dont have an account
> > there.  also is there tools for mirroring cvs from servers cause that
> > could be another option.  i got autoconf, autogen stuff etc already set
> > up.  
> > 
> ok, so here are my plans. This is based on the idea that papyrus work is
> going to be integrated directly in libgda/gnome-db CVS [1].
> 
> * currently in gda-report, there's only the framework for running the
> reports through the report engine:
> 
> 	string runDocument (in string xml);
> 
> this just gets the XML file, and fills in with data, and returns the
> same XML with the data in it. For managing this XML file, I've started a
> GdaReportDocument class, which will let programs easily manage the XML
> format. We could just have a method on the GdaReportDocument class
> (print) which generates the gnome-print format from the XML, but with no
> libgnomeprintui use at all, to avoig GTK dependence

The complex bit is rendering the report to the UI, not so much the
extracting of data.  executing the langauge would have to be server side
and run in the first phase here when it generated the xml file.  

 
> * XML format: I don't really care about the XML format we're going to
> use as long as it fits all our needs. So please, Carlos and Andrew, just
> get along on the format you both want.
> 
> * datasource independence: as I said, I can easily add a CORBA interface
> for adding new datasources to the report engine, and a C wrapper for it
> in libgda-report. This, as said before, will work by using a prefix
> (gda:, xml:, etc) in the command to be executed. I'd prefer to use CORBA
> interfaces for this rather than normal plugins (.so), since it makes
> more sense to use the glue used everywhere in libgda.

just put the corba interfaces in the binding libary, there is no
bindings for corba i know of for talking directly with postgresql.  This
is just to give more flexibility for porting (orbit doesn't currently
work under win32), and future proofing. and hey the works already done. 
plus i need this reporting system to talk to some oldschool databases of
mine which dont have corba bindings.
 
> * UI: this is the part that would use the preview stuff from
> libgnomeprintui. Also, we need a designer, which is on what Jim will be
> working on. This should be in libgnomedb

At the moment payprus covers both the UI and the low level stuff.  I can
make sure they stay reasonbly seperate.  but as i said before the
wouldbe outputed xml file is stored in memorory at the moemnt in data
structures. the phases it goes though in report generation are as
follows:

parse xml, generate strcutres reprensenting it.  connect to db's etc.
generate a layout, defining hwo the objecst correspond to each other
get the data, create a render object.
<- here a step could be added to generate an xml file from the render.
(this would be taking the render_item structure present and creating an
xml file from it, and then a function to re-generate the structure
again)
render_print lays the stuff out in 2d space.
print sends the stuff to libgnomeprint



> so, if this fits you, please have a look at how to integrate your
> current code into the libgda/libgnomedb sources (CVS HEAD, for GNOME 2),
> and tell us, so that we can start integrating it. If it does not fit you
> (or somebody else), please add/remove/fix what I said.
> 
> If it fits you, I'll continue my work on the report engine (runDocument)
> and add the stuff for the different data sources to be supported.

I suppose now we have to get familure with each others code.
 
> cheers
> 
> [1] send an email to accounts gnome org, cvsmaster gnome org and CC it
> to me, and in the message just say that you're going to help us in
> gnome-db, and that you want a CVS account
> -- 
> Rodrigo Moya <rodrigo gnome-db org> - <rodrigo ximian com>
> http://www.gnome-db.org/ - http://www.ximian.com/