Re: [xml] setting URL for xmlRelaxNGParserCtxt?

[Martijn Faassen <faassen infrae com>]

Hey Daniel,

I hope you'll still answer the other part of my mail (the Relax NG 
include processing errors).. that's a bit more urgent right now. :)

Daniel Veillard wrote:
> On Wed, Jan 26, 2005 at 11:35:52AM +0100, Martijn Faassen wrote:
>
> > > One note though, I'm using dictionnaries more and more intensively those
> > > days, so keeping a single dictionnary for everything, while it clearly
> > > speed-up processing may grow quite a bit for example as you process random
> > > documents. It might be a good idea to "sometimes" reset the dictionnary
> > > to avoid explosion, for example for all the ID strings ever processed 
> > > by the program.
> >
> > I don't use a single dictionary to speed up processing. I am using a 
> > single dictionary as I have absolutely no way to know which nodes from 
> > which nodes are going to be combined. I actually don't know what these 
> > dictionaries are used for (are elements stored in them? text node 
> > strings? both? and why? :), but having separate ones is just an 
> > impediment to moving nodes around freely. Moving nodes around freely is 
> > absolutely essential in my library's case, as the API I'm implementing 
> > allows this always.
>
>   okay then just keep the same dictionnary, but it may grow indefinitely
> and may slow things down after a while.

So what is the dictionary used for right now? I don't want things to 
slow down after a while, obviously.

> > What do you mean by resetting for every ID string? You mean as soon as 
> > an ID is encountered in an XML document? This may never happen for many 
> > classes of XML document, right?
>
>   IDs values are stored in the dictionnary, they are potentially random
> strings, and hence may grow your dictionnary indefinitely even if the
> documents you manipulate have a fixed vocabulary.
>
>   If it doesn't make sense for you, just forget about it.

Well, no, I'd rather understand it so I can fix my code and *then* 
forget about this. I also am interested in dictionaries to see whether 
there's any smart Python unicode object caching possible.

Dictionaries won't grow indefinitely if I throw away nodes once every 
while, right? The refcount will go down of dictionary values, to 0 if 
it's a unique string, and the thing (tag name? text node value?) will go 
away.

Now understand that there are "ID"s which are potentially random 
strings, and will grow my dictionary. Will those IDs never be removed 
because they're not dereffed when a document goes away? Are these XML 
per-document unique IDs we're talking about, special attributes which 
typically get declared to be an ID in a DTD? I'm asking as I haven't 
done a lot with IDs yet.

Regards,

Martijn