Re: Sun's fopen() fd limit of 256

[Joseph Kowalski <Joseph Kowalski eng sun com>]

> I'm not sure if everyone is aware, but Sun has a long standing issue where
> fopen will fail if the # of open fds is greater than 0xff. The underlying
> reason is  that they never updated their FILE struct to change the UNIX fd
> from an unsigned char. Only if you build your app as _LP64 does it finally
> use an int for the fd.
> 
> In order to work around this bug we've maintained our own imitation f* API for 
> reading/writing that does not have this 256 fd limit.
> 
> Is there any benefit to providing a full GLib fopen/fread/fwrite/etc API which 
> provides its own implementation if the system implementation has an fd 
> limitation? The configure script could detect this and enable it if necessary. 
> Just wondering if anyone else had run into this problem...
> 
> Thanks,
> 
> Andrew Paprocki
> Bloomberg LP

Disclaimer: I'm a long time SunOS engineer and a short time GNOME lurker.

The reason "they never updated their FILE struct" is that ol' binary
compatibility constraint.  I'm sure you are all aware that Sun goes to
great lengths to keep existing binaries working.  Right or wrong, that's
the commitment we make to our customers.

The core of the technical problem, is that getc/putc historically are
macros.  Every application that uses these has built in a hard dependency
on the *exact* layout of the file structure (or at least the relevant sections
of it).  There also are some interesting details in the ABIs which apply
constraints here.

Depending on how one would try to finesse the implementation, one gets
different exact set of restrictions, but they all boil down to being unable
to mix legacy binaries (shared objects) with the "newer" implementation.

Understanding this, it is possible to implement many flavors of "90%
solutions" (and please, lets not start proposing them to this broad
alias). That brings up the question as to if a "90% solution" is worthwhile.  
That is for you to judge, but be prepared to write a very long and twisted
manual page as to when that "solution" will work for you and be prepared
to see a number of unexpected failures because dependencies lurk where
you least expect them.

We (Sun) spent a lot of time looking for a 100% solution and spent a
incredible amount of resources to even provide the 64-bit solutions
(circa 1995-2000).  I take solace in trend that 32-bit simply won't
matter in a few years.

Oh yea, this message reflects my personal views and not those of Sun
Microsystems.

- Joseph Kowalski