Moving along with PyMongo

In a previous post, I introduced MongoDB and its Python driver pymongo. Here, I continue in the same vein, describing in more detail how you can become productive using pymongo.

Getting Started with MongoDB and Python

If you've been following this blog for a while, you've seen me mention MongoDB more than once. One exciting thing for me is that I'll be co-teaching a tutorial at PyCon this year on Python and MongoDB that will cover MongoDB, PyMongo, and Ming. So to hopefully whet your appetite for learning more at the tutorial, I thought I'd write a few posts covering MongoDB, PyMongo, and Ming from a beginner's perspective.

MongoDB's Write Lock

MongoDB, as some of you may know, has a process-wide write lock. This has caused some degree of ridicule from database purists when they discover such a primitive locking model. Now per-database and per-collection locking is on the roadmap for MongoDB, but it's not here yet. What was announced in MongoDB version 2.0 was locking-with-yield. I was curious about the performance impact of the write lock and the improvement of lock-with-yield, so I decided to do a little benchmark, MongoDB 1.8 versus MongoDB 2.0.

MongoDB Rocks My World

I am delighted to announce that I'll be speaking at several MongoDB-related events over the next couple of months. So if you're in or near Dallas, Seattle, or Silicon Valley, I'd love it if you could make it to one of my talks. 10gen (the makers of MongoDB) do a great job of putting on regional conferences that are extremely reasonably priced (typically $30 student and $50 early bird), and there's usually a nice after-party for each one, so there's no excuse not to go.

So anyway, here are the conferences I'll be speaking at:

• November 17: MongoDallas 
• December 1: MongoSeattle 
• December 9: MongoSV (Santa Clara, CA) 

I also wanted to say a few words on why I enjoy using MongoDB and why it is such a good fit for a lot of the problems I face...

MongoDB Atlanta User Group

So last night I had the opportunity to speak about Zarkov at the Atlanta MongoDB user group (not to be confused with the MongoATL conference). It was a good crowd, and we had a good time munching on pizza beforehand and talking about MongoDB afterward. Anyway, I did a little realtime demo combining some stuff I've been doing with Socket.io, Zarkov, and Flot and promised I'd make the code available. A mostly-functional version is available at http://sf.net/u/rick446/mongodb-atl/.

Also, I figured I'd use this post to collect all the presentation paraphernalia from last night, so below are the slides and somewhat poor quality video I recorded. Hope you manage to enjoy it anyway!

WebSockets to Socket.io

In a previous blog post, I showed how you can use Gevent, ZeroMQ, WebSockets, and Flot to create a nice asynchronous server that graphs events in real time to multiple web consumers. Unfortunately, Chrome is just about the only browser that my demo worked in due to some issues with the WebSockets standard. So I've updated the example to use Socket.io, a WebSockets-like Javascript library that will work across multiple browsers.

Extending Zarkov's Map/Reduce

Since I've been working on Zarkov, I've been writing a few
map/reduce jobs. One of the things I noticed about map/reduce, at
least as I've implemented it in Zarkov, is that it's pretty
inefficient if you want t o generate several aggregate views of the
same data (like say an event stream). In order to meet Zarkov's
performance requirements without making my operations team
too angry, I decided to "extend" map/reduce, as it's
implemented in Zarkov. I'm not terribly creative, so I called this
command xmapreduce. Here' I'll briefly describe
map/reduce, show the problem with it, and explain the solution
implemented in Zarkov.