The primary functionality Serveez provides is a framework for Internet services. It can act as a kind of server collection and may even replace super-servers such as the inetd daemon.
Its key features and benefits are:
Serveez’s I/O strategy is the traditional
select method. It is
serving many clients in a single server thread. This is done by setting
all network handles to non-blocking mode. We then use
to tell which network handles have data waiting. This is the
traditional Unix style multiplexing.
An important bottleneck in this method is that a
sendfile from disk blocks if the data is not in core at the
moment. Setting non-blocking mode on a disk file handle has no effect.
The same thing applies to memory-mapped disk files. The first time a
server needs disk I/O, its process blocks, all clients have to wait, and
raw non-threaded performance may go to waste.
select is limited to the number of
FD_SETSIZE handles. This limit is compiled into the standard
library, user programs and sometimes the kernel. Nevertheless, Serveez
is able to serve about one thousand and more clients on GNU/Linux, a
hundred on Win95 and more on later Windows systems.
We chose this method anyway because it seems to be the most portable.
An alternative method to multiplex client network connections is
poll. It is automatically used when ‘configure’ finds
poll to be available. This will work around the builtin (g)libc’s
select file descriptor limit.
The limits set by
See the Solaris FAQ, question 3.45.
Use sysctl -w kern.maxfiles=nnnn to raise limit.
See Bodo Bauer’s /proc documentation. On current 2.2.x kernels,
echo 32768 > /proc/sys/fs/file-max
increases the system limit on open files, and
ulimit -n 32768
increases the current process’ limit. We verified that a process on Linux kernel 2.2.5 (plus patches) can open at least 31000 file descriptors this way. It has also been verified that a process on 2.2.12 can open at least 90000 file descriptors this way (with appropriate limits). The upper bound seems to be available memory.
On Win9x machines, there is quite a low limit imposed by the kernel: 100 connections system wide (!). You can increase this limit by editing the registry key HKLM\System\CurrentControlSet\Services\VxD\MSTCP\MaxConnections. On Windows 95, the key is a DWORD; on Windows 98, it’s a string. We have seen some reports of instability when this value is increased to more than a few times its default value.
More than 2000 connections tested. It seems like the limit is due to available physical memory.
One of the problems with the traditional
select method with
non-blocking file descriptors occurs when passing a large number of
descriptors to the
select system call. The server loop then
goes through all the descriptors, decides which has pending data, then
reads and handles this data. For a large number of connections (say,
90000) this results in temporary CPU load peaks even if there is no
Along with this behaviour comes the problem of “starving” connections.
Connections which reside at the beginning of the
select set are
processed immediately while those at the end are processed significantly
later and may possibly die because of buffer overruns. This is the
reason why Serveez features priority connection: it serves listening
sockets first and rolls the order of the remaining connections. In this
way, non-priority connections are handled in a “round robin” fashion.
Other server implementations solve these problems differently. Some
start a new process for each connection (fully-threaded server) or split
select set into pieces and let different processes handle
them (threaded server). This method shifts the priority scheduling to
the underlying operating system. Another method is the use of
asynchronous I/O based upon signals where the server process receives a
signal when data arrives for a connection. The signal handler queues
these events in order of arrival and the main server loop continuously
processes this queue.