Tuesday, August 07, 2012

Building TCP Servers with Gevent

Continuing on in my series on gevent and Python, this article discusses how to build TCP servers using the infrastructure provided with gevent. If you're just getting started with Gevent, you might want to read the previous articles in this series first:

And now that you're all caught up, let's jump into gevent servers...

Basic TCP servers with Gevent StreamServer

Setting up a tcp server in Gevent extremely straightforward:

  • You create some handler function that will perform communication over a connected TCP socket.
  • You create an instance of gevent.servers.StreamServer, passing your handler function in its constructor.
  • You start the server.

And that's it! By default, each socket gets its own greenlet. Here's a simple example of an echo server we could set up:

from gevent import socket
from gevent.server import StreamServer

def handle_echo(sock, address):
    fp = sock.makefile()
    while True:
        line = fp.readline()
        if line:
            fp.write(line)
            fp.flush()
        else:
            break
    sock.shutdown(socket.SHUT_WR)
    sock.close()

server = StreamServer(
    ('', 1234), handle_echo)

server.serve_forever()

The only thing here that bears explanation is the fp = sock.makefile(). What this does is create a file wrapper around a socket so we can use our normal readline, write, and flush methods, pretending we're not really dealing with a socket at all.

Limiting concurrency

One of the nice things about gevent is that it lets you handle many more simultaneous connections than a threaded implementation of a server would be able to. In some cases, however, you might still want to limit the number of greenlets started by a StreamServer. For those cases, you can either pass an instance of gevent.pool.Pool or an integer as the spawn argument:

server = StreamServer(
    ('', 1234), handle_echo,
    spawn=10000) # limit to 10,000 simultaneous connections

Customizing the listening socket

The StreamServer also supports several other features. For instance, if we wish to change the backlog argument to listen() on the listening socket, we can simply pass backlog to the StreamServer constructor:

server = StreamServer(
    ('', 1234), handle_echo,
    backlog=1000)

If we want to do further customization, we can simply create the StreamServer with an already-prepared and listening socket instead of the (IP, port) we were using above:

sock = socket.socket()
sock.bind(('', 1234))
sock.listen(256)
server = StreamServer(
    sock, handle_echo)
server.serve_forever()

Encrypting your connections

One of the reasons you might consider create the listening socket is to provide an encrypted SSL socket using gevent.ssl.socket. This, however, will not work due to some implementation details in the ssl module. To create an SSL stream server, then, you would simply pass the SSL information to the SocketServer constructor:

server = StreamServer(
    ('', 1234), handle_echo,
    keyfile='server.key', certfile='server.crt')

Conclusion

And that's pretty much all there is to it. Building gevent-based TCP servers is really pretty simple, and it's nice to know you can scale to thousands of connections without overloading your server. So what do you think? Have you ever built any socket servers that you'd think about porting to gevent? Any other use-cases you've run into? I'd love to hear about it in the comments below!

5 comments:

  1. "One of the reasons you might manually create the listening socket is to provide an encrypted SSL socket using gevent.ssl.socket"

    Passing encrypted socket as a listener won't work, keyfile/certfile is the way to create SSL sockets.

    The reason it won't work lies in the implementation. StreamServer implementation is non-blocking, so it uses non-blocking accept(), however SSLSocket's accept() cannot be non-blocking because it involves a doing handshake.

    Otherwise, nice introduction.

    ReplyDelete
    Replies
    1. Thanks for the correction! I'll make the change.

      Delete
  2. I confess I don't understand the purpose of this idiom:

    while True:
    ``if predicate:
    ````something()
    ``else:
    ````break

    Couldn't that be replaced by:

    if predicate:
    ``something()

    Even if you mean to continue instead of break, the else clause is redundant. Please let me know if I'm missing something here. I've really enjoyed these gevent articles, even if I'm mildly annoyed that blogger comments won't accept a <CODE> tag while blithely munching whitespace.

    ReplyDelete
    Replies
    1. Never mind, I'm a moron. You loop until readline returns an empty string.

      Delete
    2. Thanks for the comments, Jess. You've got the correct meaning, though it might've been more succinct if I'd written:

      while True:
      ``something()
      ``if not predicate: break
      ``something_else()

      Anyway, thanks for the close reading of the code examples!

      Delete