Re: poll(2) only can handle 256 fds?

[Markus Lausser <sgop users sourceforge net>]

On Fri, 2005-03-11 at 13:41, Gus Koppel wrote:
> Iago Rubio wrote:
>
> > > > i get the message
> > > >   *** GLib *** : poll(2) failed due to: Invalid argument.
> > > > thousand times when running my application.
> > > > It seems that it happens if i have accepted 100 < x < 200
> > > > tcp connection from which i listen for input.
> > > >
> > > > What can i do? I am using glib 2.6.0 on a linux system.
> > >
> > > I've did some investigation, and it appears that glib runs into
> > > an endless loop when it tries to poll more than 256 fds at once.
> >
> > It makes sense ... read /usr/include/linux/limits.h 
> >
> > #define OPEN_MAX         256    /* # open files a process may have */
> >
> > It's not glib, but your kernel.
> >
> > If you need to open more files in a single process, change this number
> > and recompile the kernel.
>
> No need for that! Speaking of the Linux kernel, OPEN_MAX is a legacy
> value without meaning any more. In Linux <= 2.2.x kernel series there
> existed a hardcoded limit of 1024 (default value) file descriptors
> indeed, but starting with kernel 2.4.x the number of file descriptors
> has become dynamic. The initial value appears to be __FD_SETSIZE,
> defined in include/linux/posix_types.h, which defaults to 1024 FDs per
> process. When a process requests to open more files (connections) than
> this, the kernel dynamically expands the file descriptor pool of the
> process up to NR_OPEN files (include/linux/fs.h), which is about 1
> million in my kernel. Since I can confirm polling of more than 1024 open
> files (TCP connections) is possible on Linux, scopes of <200 must be a
> limitation or bug in glib or somewhere else in Userland.
>
> However, is it possible that you're just suffering from some lower
> ulimit settings (command 'ulimit'), which may be limiting the resources
> available to your program?

$ ulimit -a
core file size (blocks)     1000000
data seg size (kbytes)      unlimited
file size (blocks)          unlimited
max locked memory (kbytes)  unlimited
max memory size (kbytes)    unlimited
open files                  1024
pipe size (512 bytes)       8
stack size (kbytes)         8192
cpu time (seconds)          unlimited
max user processes          14336
virtual memory (kbytes)     unlimited

$ uname -sr
Linux 2.4.2-2

getrlimit(RLIMIT_NOFILES, ...) reports 1024 for both the current
and the max value. Also using setrlimit() doesn't solve the problem.

I've tried another application which also uses poll() (and no glib),
and it can process an array containing more than 256 entries.

But this little test program also fails here with more than 256 entries:
-----------------------
#include <stdio.h>
#include <stdlib.h>
#include <sys/poll.h>

static struct pollfd* create_fds(int num) {
  struct pollfd* poll = NULL;
  int i1;
  
  poll = malloc(sizeof(struct pollfd) * num);
  for (i1 = 0; i1 < num; i1++) {
    poll[i1].fd = 0;
    poll[i1].events |= POLLIN;
    poll[i1].revents = 0;
  }
  return poll;
}

int main(int argc, char** argv) {
  struct pollfd* polls = NULL;
  int num, res;

  if (argc != 2) {
    printf("*** one integer argument required ***\n");
    return -1;
  }
  num = atoi(argv[1]);

  polls = create_fds(num);
  res = poll(polls, num, 5000);
  printf("poll() returned %d\n", res);
  free(polls);

  return 0;
}
-----------------------


Markus.