Wednesday, August 08, 2012

Building Web Applications with Gevent's WSGI Server

Continuing on in my series on gevent and Python, this article discusses how to use gevent to power your Python WSGI web applications. If you're just getting started with Gevent, you might want to read the previous articles in this series first:

And now that you're all caught up, let's jump into gevent's WSGI support...

WSGI refresher

For those not familiar with the Python web server gateway interface (WSGI), I'll provide a very abbreviated intro here. In WSGI, your application consists of a single function that takes environ and start_response arguments. That function will be called once for each web request received by your server. The environ argument is a Python dictionary that holds information about the request and about the server software itself. The start_response argument is a function that your application should call to set status and headers on the HTTP response.

Once your application has called start_response, you can either return an iterable such as a list, or simply begin yielding strings to send back as the body of the response. A simple "hello world" WSGI application might look like the following:

def hello_world(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/html')])
    return [ '<b>Hello world!</b>\n' ]

If you prefer to use the yield statement instead, your application might look more like this:

def hello_world(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/html')])
    yield '<b>Hello world!</b>\n'

The difference between the two approaches is that if you yield strings a little at a time, you can send a large amount of data back to the client without needing to buffer it all in memory at once if the WSGI server you're using supports it.

Running your WSGI app inside gevent

Gevent actually includes two separate servers capable of calling Python WSGI web applications, located in the gevent.wsgi and gevent.pywsgi modules:

  • gevent.pywsgi has a WSGI server implemented natively in gevent, and it supports streaming responses, HTTP pipelining, and SSL.
  • gevent.wsgi has a WSGI server based the HTTP server in libevent, so it's quite a bit faster than pywsgi, but it doesn't support streaming responses, HTTP pipelining, nor SSL.

If we want to take our WSGI application above and wrap it in the pywsgi server, the approach is quite similar to a "regular" TCP StreamServer discussed in the previous article, except that our WSGI application serves the purpose of the handler in the StreamServer case. Our entire application, then is the following:

from gevent import pywsgi

def hello_world(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/html')])
    yield '<b>Hello world!</b>\n'

server = pywsgi.WSGIServer(
    ('', 8080), hello_world)

server.serve_forever()

Now we can request a page and see that everything's working just fine.

$ curl http://localhost:8080
<b>Hello world!</b>

wsgi versus pywsgi

If we run curl with the -v argument, we can see the headers sent back by the server. We can observe what the WSGI server's doing by paying attention to the lines starting with < and *:

$ curl -v http://localhost:8080
...
< HTTP/1.1 200 OK
< Content-Type: text/html
< Date: Sat, 04 Aug 2012 19:34:51 GMT
< Transfer-Encoding: chunked
< 
* Connection #0 to host localhost left intact
* Closing connection #0

There are a couple of things of interest here:

  • The Transfer-Encoding: chunked indicates that we can stream out data a little bit at a time, without having to buffer everything in server memory.
  • The next-to-last line indicates that the connection is left intact. This means that the server supports HTTP pipelining.

If we change our code just a bit to use the wsgi module instead (leaving everything else the same), we'll see a different set of headers:

$ curl -v http://localhost:8080
...
< HTTP/1.1 200 OK
< Content-Type: text/html
< Content-Length: 19
< Server: gevent/0.13 Python/2.7
< Connection: close
< Date: Sat, 04 Aug 2012 19:41:49 GMT
< 
* Closing connection #0

The points important to note here include:

  • The Content-Length: 19 header is present, and the Transfer-Encoding: chunked is missing. This is our clue that the server has buffered the entire response in memory before beginning to respond.
  • The wsgi server is quite forthcoming about its identity with the Server line, going so far as to say which version of gevent and Python are being used. This is a potential security problem, as it allows an attacker to target vulnerabilities to the particular server software used.
  • The Connection: close header and the lack of anything indicating that the connection was left intact indicates that HTTP pipelining is not supported.

Given the functionality differences between the servers illustrated by the headers they return, you might wonder why you'd ever use wsgi instead of pywsgi. The answer lies in the performance difference between the two servers. In my testing, wsgi could handle 3800 requests per second, while pywsgi could handle around 2400 requests per second, for a speedup of 1.59. (For those interested, that's using ab -n 10000 -c 1000 http://localhost:8080/ as a micro-benchmark.)

So if you have an application where the performance is limited by the WSGI server overhead, you might want to consider using the wsgi server. In my experience, however, even fairly trivial Python WSGI applications have performance that is measured (at best) in the hundreds of requests per second. Put another way, the difference in overhead introduced by wsgi versus pywsgi is less than 160 microseconds, and you give up quite a bit of functionality in the process.

Using Python web frameworks with gevent

In most cases, you'll be using a web framework to build your larger Python web applications. In most if not all cases, those frameworks provide a WSGI application that you can plug directly into gevent. Some of the relevant links are below:

  • Django - In particular, note that projectname/wsgi.py contains a WSGI application that you can use with gevent's WSGIServer.
  • Pyramid - Pyramid provides a make_server command which is used in the documentation, but you can as easily use the WSGIServer from gevent.
  • Flask actually has gevent instructions right on the web page.
  • TurboGears - Since TurboGears uses PasteDeploy, you can use paste-gevent to wrap your application.

In most cases, you don't need to make any significant changes to your web applications to make it work correctly with gevent, particularly if you use the gevent monkey-patching module.

Conclusion

Using gevent as a Python WSGI server has one great side-effect: you can spin off greenlets to do background processing whenever you want. Of course, it's also nice to be able to handle thousands of simultaneous connections without seriously taxing your server. One place where this is useful is in web socket or long-polling applications where you need to support lots of simultaneous connections.

There are a couple of modules, gevent-websocket and gevent-socketio, that make this type of application work well in a gevent WSGI wrapper. I've mentioned them in previously, but I'll be going into some more depth in upcoming articles, so watch out for them.

So what do you think? Do you already use the gevent server to host your Python WSGI applications? Is it something that you'd consider doing? Anyone using gevent for long-lived web clients and not using websockets or socketio? I'd love to hear about it in the comments below!

14 comments:

  1. I'm new to all this. How is gevent's WSGI server related to Gunicorn? Does one use the other? Are they 'competing' products?

    ReplyDelete
    Replies
    1. Gunicorn is more of a "server manager" that spawns multiple worker processes, each of which could be a) synchronous, b) async (using gevent), or c) async (using tornado). In general, the gevent server will probably be easier to initially set up and configure, but on a multicore system, you'll probably get better performance out of gunicorn (possibly with gevent workers). (See http://gunicorn.org/design.html for details on how to choose worker types in Gunicorn).

      Delete
    2. a way to utilize multicore when running with gevent: run more than one python interpreter-- though it likely takes some forethought in program-design.

      Delete
    3. @scape: Thanks for the comment! I believe that's exactly the approach used with gunicorn+gevent (one interpreter monitors multiple worker processes, each of which use gevent for async networking and greenlets).

      Delete
    4. We want to provide developers with ourServerApp (= ourApp + gevent.pywsgi). They should be able to run this configuration as-is for development and testing.

      For production, they can add (around ourServerApp) their preferred wsgi (application) webserver (including gunicorn, django etc.) and a 'proper' web server such as nginx at the front.

      Is this possible?

      Delete
    5. I don't see any reason why you wouldn't be able to do this, so long as you don't write ourServerApp with any assumptions about what server it's running under (for instance, using any of the gevent library methods would probably not be compatible with running under a multithreaded WSGI server).

      Delete
  2. Cool!
    now I know how to write HTTP streaming server for testing purposes
    thanks a lot for post !!!

    ReplyDelete
    Replies
    1. Thanks, Sergey! Glad you found it useful!

      Delete
  3. how to serve the static pages .. ?? is there a method in which gevent can be run as standalone server for both wsgi and its static contents.

    ReplyDelete
    Replies
    1. Hi Rakesh,

      You can certainly use the gevent wsgi server to serve static pages; I have an example at http://blog.pythonisito.com/2012/07/realtime-web-chat-with-socketio-and.html if you're interested. In general, however, for a production system it's better to use a dedicated static content server or a CDN. Most web frameworks also provide facilities for serving static content (and most Python WSGI framework can be run inside the gevent wsgi servers).

      Thanks for the comment!

      Delete
  4. This is a great write up, thank you. I think that version changes are conspiring against me when trying to use these techniques - I'm getting this error:

    AttributeError: GreenSocket has no such option: _GREENSOCKET__IN_SEND_MULTIPART

    Google isn't much help - I'm getting one result, which is a bug report for locust. The maintainer simply removed gevent_zmq from the build.

    I tried using zmq directly, but we get a lockup in gevent.sleep in zmq_producer - gevent.sleep never returns.

    Any ideas how best to proceed?

    ReplyDelete
    Replies
    1. Hmm, that's strange -- I'm using the following versions right now without any errors. It does sound like you probably have some kind of a version mismatch. (I don't think I'm actually using zmq in my prod site, though, so that might explain it.)

      pastegevent==0.1
      gevent==0.13.8
      gevent-zeromq==0.2.5

      Delete
  5. The exception is raised by pyzmq 13.x, see https://github.com/locustio/locust/issues/58.
    Remove pyzmq and reinstall it explicitely (download https://pypi.python.org/packages/source/p/pyzmq/pyzmq-2.2.0.1.zip#md5=31d4100d62e352e5e19824ded45aaac9 and run setup.py) will fix the problem...

    ReplyDelete