Fuck me on GayHub

# Let's build a WSGI server

Let’s build a wsgi server ground up with sockets, multi-processing or Select/Poll/Epoll. to achive this, you need familiar with Python’s socket programming, Python’s cucurreny model, Linux Non-blocking I/O.

WSGI(web server gateway interface) is a protocol in Python web programming, it defines how web application and web server communicate. with this protocol, you can chose various combinations about web servers and web frameworks which are wsgi-compatiable. for example, you can use Gunicorn + Django, or even Bjoern + Flask etc…

### What’s WSGI Server

WSGI server is the server side wsgi protocol implementation. which is responsible accepting connections and invoke web application to process the result though wsgi protocol, and finally return the result back to clients. with wsgi server concentrate on accepting connections, wsgi framework(application) can focus on your bussiness logic.

To summarize, as an wsgi server, it must contains following info:

• compose env variables which will be used in web framework

• start_response callable which will be called in web framework

for more detail, please reference: PEP 333: Python web server gateway interfaces v1.0

### Simple TCP echo server

As classic socket programming said, a socket server needs to bind/listen/accept/recv/send/close, a socket client needs to connect/send/recv/close. here is a simple example:

### A simple wsgi server

The simple wsgi server will listen on port, waitting for incoming connection to accept, for each connection, wsgi server will invoke web framework process(which is the main entrance implemented in web framework) the request and wait until web framework finish processing it and return the result to client. here is a simple example:

### Concurrent with multi-processing

The upper wsgi server can process a request at a time, cuz it is a single process and will block in while loop, as following code shows:

handle_one_request will block the while loop while processing the current connection. the time spend on blocking depends on your web application’s corresonding method’s processing speed.

if the first connection didn’t finished, the second connection will be blocked by the first one. how can we let the wsgi server handle multiple connections at the same time? the answer is multi-processing.

here is an example of multi-processing version, which will fork an new python process for each incoming connections.

### Concurrent with Linux’s Non-blocking I/O

The multi-processing version still have problems, like when there are massive connections, the server will create massive processes for each connection. this will exaust the server resources, and also process-switching is expensive. to achive that, you will think using process pool.

But let’s think from another side, the server maily handles I/O tasks, i.e waitting socket to be readable, read the data and process it, and write to socket. for this kind of task, we use Non-Blocking I/O.

Suppose you’re a webserver. Every time you accept a connection with the accept system call (here’s the man page), you get a new file descriptor representing that connection.

If you’re a web server, you might have thousands of connections open at the same time. You need to know when people send you new data on those connections, so you can process and respond to them.

You could have a loop that basically does:

The problem with this is that it can waste a lot of CPU time. Instead of spending all CPU time to ask “are there updates now? how about now? how about now? how about now?“, instead we’d rather just ask the Linux kernel “hey, here are 100 file descriptors. Tell me when one of them is updated!“.

The 3 system calls that let you ask Linux to monitor lots of file descriptors are poll, epoll and select. Let’s start with poll and select because that’s where the chapter started.

I am not gonna talk the details between select/poll/epoll, you can read this post for more: async io on linux select/poll/epoll.

Python’s select module have support for Linux’s select/poll/epoll, let’s use select rewrite our wsgi server:

we use following code tell os that give the readable and writable file descriptors when possible:

and then we loop the readables, if it is a listen_socket, we accept it. if it is a socket with income data, we read it.

### Summary

Let’s benchmark our wsgi server with following web application, also against Gunicorn:

to benchmark, we use wrk as following:

Gunicorn config:

Here is the result for multi-processing version:

Here is the result for Non-Blocking IO version:

Here is the result with Gunicorn multi-processing version

here is the result on my Linux book:

Single Process Multi-Process Non-Blocking I/O Gunicorn
QPS todo 739 4128 3613

And found an interesting thing, Non-Blocking IO will use one process but also can support high qps, at the same time, it takes less CPU than Gunicorn pre-fork model.