Next: , Previous: , Up: Top   [Contents][Index]

2 Concept

2.1 Overall concept

The primary functionality Serveez provides is a framework for Internet services. It can act as a kind of server collection and may even replace super-servers such as the inetd daemon.

Its key features and benefits are:

2.2 I/O Strategy

Serveez’s I/O strategy is the traditional select method. It is serving many clients in a single server thread. This is done by setting all network handles to non-blocking mode. We then use select to tell which network handles have data waiting. This is the traditional Unix style multiplexing.

An important bottleneck in this method is that a read or sendfile from disk blocks if the data is not in core at the moment. Setting non-blocking mode on a disk file handle has no effect. The same thing applies to memory-mapped disk files. The first time a server needs disk I/O, its process blocks, all clients have to wait, and raw non-threaded performance may go to waste.

Unfortunately, select is limited to the number of FD_SETSIZE handles. This limit is compiled into the standard library, user programs and sometimes the kernel. Nevertheless, Serveez is able to serve about one thousand and more clients on GNU/Linux, a hundred on Win95 and more on later Windows systems.

We chose this method anyway because it seems to be the most portable.

An alternative method to multiplex client network connections is poll. It is automatically used when ‘configure’ finds poll to be available. This will work around the builtin (g)libc’s select file descriptor limit.

2.2.1 Limits on open filehandles

Any Unix

The limits set by ulimit or setrlimit.


See the Solaris FAQ, question 3.45.


Use sysctl -w kern.maxfiles=nnnn to raise limit.


See Bodo Bauer’s /proc documentation. On current 2.2.x kernels,

echo 32768 > /proc/sys/fs/file-max

increases the system limit on open files, and

ulimit -n 32768

increases the current process’ limit. We verified that a process on Linux kernel 2.2.5 (plus patches) can open at least 31000 file descriptors this way. It has also been verified that a process on 2.2.12 can open at least 90000 file descriptors this way (with appropriate limits). The upper bound seems to be available memory.

Windows 9x/ME

On Win9x machines, there is quite a low limit imposed by the kernel: 100 connections system wide (!). You can increase this limit by editing the registry key HKLM\System\CurrentControlSet\Services\VxD\MSTCP\MaxConnections. On Windows 95, the key is a DWORD; on Windows 98, it’s a string. We have seen some reports of instability when this value is increased to more than a few times its default value.

Windows NT/2000

More than 2000 connections tested. It seems like the limit is due to available physical memory.

2.3 Alternatives to Serveez’s I/O strategy

One of the problems with the traditional select method with non-blocking file descriptors occurs when passing a large number of descriptors to the select system call. The server loop then goes through all the descriptors, decides which has pending data, then reads and handles this data. For a large number of connections (say, 90000) this results in temporary CPU load peaks even if there is no network traffic.

Along with this behaviour comes the problem of “starving” connections. Connections which reside at the beginning of the select set are processed immediately while those at the end are processed significantly later and may possibly die because of buffer overruns. This is the reason why Serveez features priority connection: it serves listening sockets first and rolls the order of the remaining connections. In this way, non-priority connections are handled in a “round robin” fashion.

Other server implementations solve these problems differently. Some start a new process for each connection (fully-threaded server) or split the select set into pieces and let different processes handle them (threaded server). This method shifts the priority scheduling to the underlying operating system. Another method is the use of asynchronous I/O based upon signals where the server process receives a signal when data arrives for a connection. The signal handler queues these events in order of arrival and the main server loop continuously processes this queue.

Next: , Previous: , Up: Top   [Contents][Index]