[xml] The NaN dilemma

[Bjorn Reese <breese mail1 stofanet dk>]

Over the last couple of weeks, I have worked together with Petr Kozelka
to find a workaround for the SIGFPE caused by the generation of NaN that
he experienced when using libxml in Kylix.

I will assume ANSI/IEEE 754-1985. The discussion may apply to other
floating-point standards, such as IEC 60559:1989, but I have not verified
this. Furthermore, the description of floating-point exceptions has been
simplified.

Whenever an arithmetic operation occurs which results in, or involves, a
NaN, the Floating-Point Unit will raise an Invalid Operation exception
(this includes 0/0 -- only if the dividend is non-zero will the Division
by Zero exception be raised).

If this exception is masked, the FPU will return NaN, which can be used
in subsequent calculations. This is the default behaviour. If the
exception is unmasked SIGFPE occurs.

Apparently, the FPU will keep signalling SIGFPE on x86 (I have not
investigated this on other processors). This causes the process to abort
with a SIGFPE, even if the signal is ignored.

As you probably have guessed by now, Kylix unmasks the Invalid Operation
exception, effectively disabling any operation involving NaN. Because
any other application could do likewise, I would like to hear comments
from others.

There only seem to be two viable solutions.

 1. We can mask the Invalid Operation exception. This has to be done
    with very compiler/platform specific code (involving assembly
    instructions). This can be done in two ways.

    1a. We can mask Invalid Operation globally (that is, masking the
        exception without restoring the original setting). This may
        induce unexpected behaviour for the application (we have to
        presume that it unmasked the exception for a reason).

    1b. We can mask Invalid Operation when we perform an operation
        involving NaN, and restore the original mask afterwards.
        This means that we must identify every arithmetic operation
        (addition, multiplication, etc.) in libxml that could involve
        NaN, and wrap these operations with the mask/restore code. I
        have not investigated if this is a multithread-safe solution
        (depends on whether or not the FPU context is shared between
        threads).

 2. We can use the C99 nan() and isnan() functions. This of course
    assumes their availability. However, we still have to ensure that
    we do not perform any arithmetic operations on NaN elsewhere in
    the library.

A third option would be to generate NaN by setting the bit-pattern for
NaN explictly (other XPath implemenations does this). There are two
problems with this approach. First, it assumes IEEE 754 hardware (and
especially mainframes are famous for having pretty weird hardware).
Second, we still cannot use NaN in arithmetic operations.

Of course, we also have the "not my problem" solution, which is to
describe that the Invalid Operation exception must be masked while
using libxml.

I am not sure what the best approach is (although I prefer solution 2
because it is the less platform dependent solution).

Whoever decided that NaN should be part of XPath should be publicly
spanked :)