|
On 06/07/2018 12:54 AM, Eric S.
Eberhard wrote:
I know I am the oddball here but -- why use DTDs
at
all?
I gave reasons above. I am working on a tool. How people using the
tool is not under my control. Maybe we can focus on the opportunity
to improve libxml2 a bit here.
I supply software to a lot of companies (thousands
through
dealers). Many exchange millions of XML docs per day. I've
used this
since it was libxml. Even have some patches in there. My
application
is proprietary (meaning XML to get an order or tell a customer
our
availability is simply XML I designed and documented and give to
my
customer's customers (via download from a Web page)). Once they
get it
working it pretty much always works. They write software to
create
orders and send them to us -- it is consistent (I know, not
everyone
has this luxury so this may not apply to everyone). So why
check them?
I also found that I was getting a gagillion support tickets
because
DTDs ... simple things like a date ... seem to escape people --
take
June 7, 2018
In our date fields we will take:
Jun 7 2018
June 7 2018
the above with commas and any case (upper/lower/mixed)
6/7/18
6/7/2018
2018/6/7
20180607
180606
06-07-18
And actually many many more. Anything that is a date goes through
this
one routine and if there is any way in the world to extract a
date, we
do.
Ditto money -- say $1,245.56
We accept:
$1,245.56
1245.56
124556 (decimal is implied at 2 places if no decimal
is
found)
1,235.56
And many more - same thing, one routine reads it and if we can
possibly
get a reasonable number, we do.
This, in turn, reduced our CONSTANT support tickets for silly
things
like a format of something to ZERO. Which I like.
Even sicker -- we ignore case on tags. All of our XML is designed
to
not use duplicate names with different cases (stupid thing to do
anyway
-- expect orderNumber and OrderNumber to both be used, as
different
things).
As long as the customer is consistent and the XML is well formed
we
scan the tree and compare tags without regard to case. A WHOLE
LOT
more support tickets gone.
A lot of the people we deal with are not sophisticated. As the
receiver of XML we decided it was much better to be as flexible as
possible and take what we can if at all possible. After all -- a
DTD
can indeed tell you if an address comes in without a city name.
And
reject it and usually generate a support ticket. Since we use an
on-line AVS system (more XML) and if we have the zip and the
address
otherwise matches ... we don't need the city and state ... the AVS
system provides it. And if it fails they will get an error back
from
us (from the application) anyway. So why use a DTD to see if the
city
or state were sent? A LOT MORE support calls removed.
And, of course, performance without the DTDs is much better.
As a result we are able to give documentation to new customers and
they
are able to get it up and running with little to no help. Any
serious
errors we cannot fix are clearly explained in the responses BY THE
APPLICATION and not by a DTD.
Being flexible on our end reduces support tickets which is all I
care.
I would rather code for all the mistakes I can think of an enduser
would make (and we add new ones when they crop up) than be strict
and
do a lot of support. We don't think DTDs are flexible enough.
And I
hate making them :-)
We do offer a page with DTDs they can use manually to check their
document if they like -- or they can send it to our test system.
Once
they are running they seem to do just fine.
As programmers it is hard to believe but sometimes it is better
for us
to make slightly less efficient code in order to make the human
aspect
much more efficient. I once had someone send me a link to a
"contest"
which was a convoluted C statement and asking to solve what the
result
would be. My response -- "fire the programmer!"
If it takes 100s of competent C programmers to get the right
answer
(and only a small percent did) to read a line of code -- it is bad
code. And for people's information, modern computers read ahead
and
pre-execute code based on all kinds of weird logic. Simple C code
is
easy for it to handle ... but convoluted code ends up stopping the
pre-execution and is actually slower -- may have less lines of
code --
but it will be slower. I see nothing wrong with short clear clean
code
with as little craziness as possible. This is the same with XML
-- one
can go overboard easily, K.I.S.S. :-)
Not being so strict and no DTDs has had other benefits -- say EDI
(from
old IBMs) -- we have a cheap program that maps EDI to XML and
back. So
we can handle EDI -- and we don't need new software (after the
conversion). We accept the EDI, convert to XML, run our standard
application, create XML response, which is converted to EDI. The
package we use is low cost and no, it won't work too well with
DTDs as
EDI has it's own problems.
I could go on but most of you have probably skipped this post by
now :-)
E
On 6/6/2018 3:00 PM, Stefan Sauer wrote:
On 05/17/2018 06:01 PM, Stefan Sauer wrote:
On 05/17/2018 04:18 PM, Nick Wellnhofer wrote:
On 16/05/2018 21:51, Stefan Sauer wrote:
So one solution could be another flag to enable this?
Yes, but it would be rather ugly.
In which sense? I guess because it is something that noone should need
to know about or have to care about?
Thanks, reading the code. Need to figure where we could cache external
subsets and what a suitable keys is (ExternalID ?).
Note that I'm currently not planning to review and integrate larger
patches from other developers. I only took over some libxml2
maintenance duties because noone else did. So even if you write a
high-quality patch, it might never get merged.
Thanks for making this clear upfront. This is how I ended up becoming
the gtkdoc maintainer :)
Caching external subsets for XIncludes certainly sounds like a nice
feature but I would prefer to find a simpler solution. For example,
can't you just omit the external DTD from included documents?
Yeah, right now, the benefit of having the DTD is that one can validate
fragments. I'll do some research (aka grepping over existing projects)
to see how the doc-type headers being used today look like. If all that
people do is using an entity to inject the version, I'll write a
migration tool.
We have a test that validates the doc, but I think I can change this to
just resolve all xincludes and check through the top-level doctype.
Just to add to this, I am assuming a lot of people follow this book
http://www.sagehill.net/docbookxsl/ModularDoc.html#UsingXinclude
and using a DOCTYPE is part of the examples.
You wrote:
and gtk-doc will replicate this for the fragments (replacing 'book' with
e.g. 'refentry'). This way one can e.g. inject things like a version.
What do you mean by "inject things like a version"? Why exactly do
your included documents have to reference an external DTD?
The documentation consists of a handwritten master doc (type book), that
includes more handwritten parts (e.g. tutorials, guides) and include
generated reference docs. When gtkdoc generated the reference docs, it
applies takes the doctype header of the master-doc as a template and
uses that for the generated reference docs. If the master doc has
entities declared, those can be expanded in the reference fragments.
Thats the part I will check how widely it is actually used.
Stefan
Another idea is to stop loading external DTDs for XIncludes without an
XPointer _expression_. This would still change the behavior for some
users but it's much less likely to cause problems.
change the behaviour, as in we would not catch validation errors?
Too bad that xmlXIncludeParseFile() does not get the parent parserCtx,
in that case we could apply the same flags'.
Nick
I definitely don't know enough about the implications here. I was mostly
thinking to see if we can stick a dictionary of <dtd-identifier,
xmlDtdPtr> into the Parser Context and before actually loading a dtd,
check if we did already and reuse. Somehow the dict needs to be stored
in the top-level doc, when parsing is done (do we need the dtds once the
doc has been parsed?). We only free the dtds with the top-level doc. But
I agree, it is not going to be a two liner.
It seems that xmldict is only handling key and value to be a string,
right? So, we'll even need out one cache data structure. I'd say it
would need to be on the _xmlXIncludeCtxt level. global is easier, but
then we can't free it ever :/
Stefan
Stefan
_______________________________________________
xml mailing list, project page http://xmlsoft.org/
xml gnome org
https://mail.gnome.org/mailman/listinfo/xml
_______________________________________________
xml mailing list, project page http://xmlsoft.org/
xml gnome org
https://mail.gnome.org/mailman/listinfo/xml
--
Eric S. Eberhard
VICS
2933 W Middle Verde Road
Camp Verde, AZ 86322
928-567-3727 work 928-301-7537 cell
http://www.vicsmba.com/index.html (our work)
http://www.vicsmba.com/ourpics/index.html (fun pictures)
|
