Just a little Python: Realtime Web Chat with Socket.io and Gevent

In a previous post, I described how to build a realtime analytics graph with socket.io. That was nearly a year ago, however, and both socket.io and the Python library gevent-socketio have both undergone significant changes. In this post, then, I'll use the latest versions of each to build a web chat server.

Requirements

Before you get started, you should make sure you have the prerequisite libraries installed. For this post, I used version 0.3.5-rc2 of gevent-socketio and version 0.9.6 of the socket.io client library. To install gevent-socketio, you can use pip:

$ pip install gevent-socketio==0.3.5-rc2

The socket.io file socket.io.min.js can be retrieved from the socketio-client Github repository.

Building the HTML

Our HTML structure is fairly simple: there is a login pane at the top to set your and a two-column chat pane where the left column displays current users and the right column displays the chat:

<h1>Socket.io Chatterbox</h1>
<div id="status" class="border">Disconnected</div>
<form id="login">
  <label for="login-input">Login:</label>
  <input id="login-input">
</form>
<div>
  <div class="border left">
    <h3>Nicks</h3>
    <ul id="nicks">
    </ul>
  </div>
  <div class="border right">
    <form id="chat">
      <label for="chat-input">Chat:</label>
      <input id="chat-input">
    </form>
    <div id="chat-data">
    </div>
  </div>

At the end of this, we have our javascripts:

<script type="text/javascript" src="/js/jquery.min.js"></script>
<script type="text/javascript" src="/js/socket.io.min.js"></script>
<script type="text/javascript" src="/js/test.js"></script>

Add a bit of minimal styling in the <head> and we're done with the HTML:

<style>
  .border { border: 1px solid black; }
  input { width: 35em; }
  .left { width: 25%; float: left; margin: 20px; }
  .right { width: 60%; float: left; margin: 20px;}
  #chat-data { height: 400px; overflow: auto; }
</style>

Building the Python server

The biggest change to gevent-socketio since my last post is the addition of namespaces. A Namespace is an object that manages a connection to a particular socket.io client. The basic idea is that the client-side socket can emit an event with data attached, triggering a handler method in the Namespace. For instance, if the client sent the login event using socket.emit('login', username), the server would call namespace.on_login(username).

Building a gevent-socketio Namespace

In our example here, we'll use the ChatNamespace class to keep a registry of ChatNamespace objects so they can communicate. To begin our namespace, we set up the registry:

from socketio.namespace import BaseNamespace

class ChatNamespace(BaseNamespace):
    _registry = {}

The next thing we can do is define initialize and disconnect methods on the namespace. These will be called when we get a new connection and when that connection dies, respectively. In our initialize method, we add ourselves to the registry and send a message to the client noting that we're connected. In disconnect, we just make sure to de-register this client from the registry:

    def initialize(self):
        self._registry[id(self)] = self
        self.emit('connect')
        self.nick = None

    def disconnect(self, *args, **kwargs):
        if self._nick:
            self._users.pop(self._nick, None)
        super(ChatNamespace, self).disconnect(*args, **kwargs)

Now we can define our event handlers. The event's we're interested in are the following:

login - This event fires when the user chooses a nickname
chat - This event fires when a chat message is received from a client

To handle login, we do a few things:

If we're already logged in under another nickname, broadcast an "exit" event noting that we're leaving the chat.
Update our nickname
Notify other clients that we have a new chat participant by broadcasting the "enter" event.
Send a "users' event to the client with a list of current chat participants

    def on_login(self, nick):
        if self.nick:
            self._broadcast('exit', self.nick)
        self.nick = nick
        self._broadcast('enter', nick)
        self.emit('users',
                  [ ns.nick
                    for ns in self._registry.values()
                    if ns.nick is not None ])

Handling "chat" events is fairly straightforward: if the user is logged in, broadcast the event (along with the nickname) to other clients. Otherwise, send a message back to the client asking them to log in:

    def on_chat(self, message):
        if self.nick:
            self._broadcast('chat', dict(u=self.nick, m=message))
        else:
            self.emit('chat', dict(u='SYSTEM', m='You must first login'))

Finally, our _broadcast method just emit()s an event to each client in turn:

    def _broadcast(self, event, message):
        for s in self._registry.values():
            s.emit(event, message)

Hooking up the Namespace to a WSGI application

Once we have our namespace, we need to actually connect it to a WSGI application. Our main WSGI app is fairly simple: if the path requested starts with '/socket.io', we'll route it to the gevent-socketio handler socketio_manage. Otherwise, we'll serve a static file. Note in particular that we're passing the ChatNamespace defined earlier to socketio_manage so it knows how to route chat messages:

def chat(environ, start_response):
    if environ['PATH_INFO'].startswith('/socket.io'):
        return socketio_manage(environ, { '/chat': ChatNamespace })
    else:
        return serve_file(environ, start_response)

Our serve_file function is fairly straightforward. (In a production system, I'd recommend serving static files from a dedicated server instead.)

def serve_file(environ, start_response):
    path = os.path.normpath(
        os.path.join(public, environ['PATH_INFO'].lstrip('/')))
    assert path.startswith(public), path
    if os.path.exists(path):
        start_response('200 OK', [('Content-Type', 'text/html')])
        with open(path) as fp:
            while True:
                chunk = fp.read(4096)
                if not chunk: break
                yield chunk
    else:
        start_response('404 NOT FOUND', [])
        yield 'File not found'

Finally, since this is a small stand-alone app, we'll go ahead and put a small WSGI server at the end of our script:

sio_server = SocketIOServer(
    ('', 8080), chat, 
    policy_server=False)
sio_server.serve_forever()

The Javascript

Our Javascript is mostly glue code, consisting of

Code to cause form submissions to send socket.io events instead
Code to respond to socket.io events

We start off by creating our socket.io connection and grabbing some elements from the DOM:

(function() {

    // Create and connect sockert
    var socket = io.connect('/chat');

    // Grab some references
    var $status = $('#status');
    var $login_form = $('#login');
    var $chat_form = $('#chat');
    var $nicks = $('#nicks');
    var $chat = $('#chat-data');

Next, we'll make our two forms (login and chat) emit socket.io events:

    // Bind the login form
    $login_form.bind('submit', function() {
        var $input = $(this).find('input');
        socket.emit('login', $input.val());
        $input.val('');
        return false;
    });

    // Bind the chat form
    $chat_form.bind('submit', function() {
        var $input = $(this).find('input');
        scroll_chat();
        socket.emit('chat', $input.val());
        $input.val('');
        return false;
    });

Next, we'll keep a list of the users so we can incrementally update it when we receive enter and exit events:

    // List of currently logged-in users
    var users = [];

Now we'll respond to some socket.io events indicating our connection status so the user gets some notification of disconnects and errors:

    // Bind events to the socket
    socket.on('connect', function() {
        $status.html('<b>Connected: ' + socket.socket.transport.name + '</b>');
    });
    socket.on('error', function() {
        $status.html('<b>Error</b>');
    });
    socket.on('disconnect', function() {
        $status.html('<b>Closed</b>');
    });

Next, we handle our enter, exit, and users events by updating our users array and rerendering the list of users in the room:

    socket.on('enter', function(msg) {
        users.push(msg);
        render_nicks();
        append_chat($('<em>' + msg + '</em> has entered the room<br/>'));
    });
    socket.on('exit', function(msg) {
        users = $.grep(users, function(value, index) {
            return value != msg });
        render_nicks();
        append_chat($('<em>' + msg + '</em> has left the room<br/>'));
    });
    socket.on('users', function(msg) {
        users = msg;
        render_nicks();
    });

Finally, handling our chat event is trivial:

    socket.on('chat', function(msg) {
        append_chat($('<em>' + msg.u + '</em> &mdash; ' + msg.m + '<br>'));
    });

To keep all the preceding code succinct, we need a few helper functions. The first one renders our $nicks chat participant list:

    // Some helper functions
    function render_nicks() {
        var result = $.map(users, function(value, index) {
            return '<li>' + value + '</li>';
        });
        $nicks.html(result.join('\n'));
    }

Next, we have a function that ensures the last line of the chat is visible:

    function scroll_chat() {
        var new_scrolltop = $chat.prop('scrollHeight') - $chat.height();
        $chat.prop({ scrollTop: new_scrolltop});
    }

Finally, our function to append chat messages makes sure that, if we're already scrolled to the bottom, we are still at the bottom when new messages come in:

    function append_chat(msg) {
        var old_scrolltop = $chat.prop('scrollTop');
        var new_scrolltop = $chat.prop('scrollHeight') - $chat.height();
        var scroll = old_scrolltop == new_scrolltop;
        $chat.append(msg);
        if(scroll) {
            scroll_chat();
        }
    }
})();

Conclusion

So that was a lot of code, but hopefully it gave you a feel for how you can bend socket.io and gevent-socketio to you will in your own applications. All the code is also available on my PyGotham SourceForge repository (the chat server was initially developed for a talk given at PyGotham). So what do you think? Is socket.io something you'd use? Do you like the new Namespace abstraction or hate it? Let me know in the comments below!

33 comments:

Markus5:24 PM
Just out of curiosity, is it possible to port the sine example to this? I implemented it with juggernaut as it seemed more straight forward...
Anonymous7:04 AM
Splendid tutorial! This is exactly what I was looking for!
Maksat1:51 AM
Guys, I've installed gevent-socketio, run example simple_chat successfully, but when i run your chat, I get this type of error:

socket = environ['socketio']
KeyError: 'socketio'

Could you please, find me out, what is going wrong...
Anonymous5:05 PM
Great post!

In your last post, a year ago, you included ZeroMQ in the mix. With the changes to Socket.IO and Gevent, how would you integrate ZeroMQ between two Socket.io servers to have multi-server chat?
Unknown11:08 PM
Hey thanks for great tutorial, it was really very educative. But I'm having the little problem with getting it to work. Im getting the IOError: [Errno 21] Is a directory error. Here's the gist https://gist.github.com/3421900, if you could help me I would be very thankfull.
Anonymous1:20 PM
Hi, Do you have any website or demo or similar that I could see? I'm just learning Python and I'm not sure if your process is what I'm looking for.

I have a python script for a chat, that connects to a local server via sockets. I run the script through the terminal. What I want is to replace the terminal with a browser page for eventual remote access for multiple users.
Thanks
Anonymous4:24 PM
Sorry for being too brief and simplistic. It is named index.html.
Anonymous4:25 PM
By the way, when trying to access without /index.html, the browser shows "Internal Server Error"
Unknown1:19 PM
awesome tut. Thanks for taking the time to do that.
Would this be called ajax based tech? or what, i'm still confides as to what ajax is. I wan to make app that takes use input computes and prints out the result below the input string, king of like ipython notebook, is that still under the ajax umbrella?
Anonymous3:18 AM
Hi Rick
This works great. I was wondering if you could show how to run this example inside a django app.
Meeks10:49 AM
Nice tutorial, Rick. You did a great job tying all the pieces together in a very small package. (And yes, I realize that I am about a year late, but it is still a helpful post.)
aya8:46 AM
Hi Rick,

I cloned sources,
installed gevent-socketio ($ pip install gevent-socketio==0.3.5-rc2),
ran: $python socketio_test.py
Open test.html in browser, and get status: discontected.
What might I miss?

Thanks!
Default Winner6:30 AM
Hello Rick,
I need to create a chat app for android and iPhone as well. Just want to know that whether socket.io will be sufficient for the purpose and if not , what architecture you would suggest .
Unknown8:10 AM
could it be that BaseNamespace has changed to Namespace?
Unknown8:14 AM
hi!
i can import only Namespace there is no BaseNamespace
like above , are they the same?