MongoDB's Write Lock

MongoDB, as some of you may know, has a process-wide write lock. This has caused some degree of ridicule from database purists when they discover such a primitive locking model. Now per-database and per-collection locking is on the roadmap for MongoDB, but it's not here yet. What was announced in MongoDB version 2.0 was locking-with-yield. I was curious about the performance impact of the write lock and the improvement of lock-with-yield, so I decided to do a little benchmark, MongoDB 1.8 versus MongoDB 2.0.

MongoDB Rocks My World

I am delighted to announce that I'll be speaking at several MongoDB-related events over the next couple of months. So if you're in or near Dallas, Seattle, or Silicon Valley, I'd love it if you could make it to one of my talks. 10gen (the makers of MongoDB) do a great job of putting on regional conferences that are extremely reasonably priced (typically $30 student and $50 early bird), and there's usually a nice after-party for each one, so there's no excuse not to go.

So anyway, here are the conferences I'll be speaking at:

• November 17: MongoDallas 
• December 1: MongoSeattle 
• December 9: MongoSV (Santa Clara, CA) 

I also wanted to say a few words on why I enjoy using MongoDB and why it is such a good fit for a lot of the problems I face...

MongoDB Atlanta User Group

So last night I had the opportunity to speak about Zarkov at the Atlanta MongoDB user group (not to be confused with the MongoATL conference). It was a good crowd, and we had a good time munching on pizza beforehand and talking about MongoDB afterward. Anyway, I did a little realtime demo combining some stuff I've been doing with Socket.io, Zarkov, and Flot and promised I'd make the code available. A mostly-functional version is available at http://sf.net/u/rick446/mongodb-atl/.

Also, I figured I'd use this post to collect all the presentation paraphernalia from last night, so below are the slides and somewhat poor quality video I recorded. Hope you manage to enjoy it anyway!

WebSockets to Socket.io

Update July 16, 2012 There have been several updates to gevent-socketio and socket.io itself. For an updated intro to these libraries, please see m y new post on Realtime Web Chat with Socket.io and Gevent

---

In a previous blog post, I showed how you can use Gevent, ZeroMQ, WebSockets, and Flot to create a nice asynchronous server that graphs events in real time to multiple web consumers. Unfortunately, Chrome is just about the only browser that my demo worked in due to some issues with the WebSockets standard. So I've updated the example to use Socket.io, a WebSockets-like Javascript library that will work across multiple browsers.

Extending Zarkov's Map/Reduce

Since I've been working on Zarkov, I've been writing a few
map/reduce jobs. One of the things I noticed about map/reduce, at
least as I've implemented it in Zarkov, is that it's pretty
inefficient if you want t o generate several aggregate views of the
same data (like say an event stream). In order to meet Zarkov's
performance requirements without making my operations team
too angry, I decided to "extend" map/reduce, as it's
implemented in Zarkov. I'm not terribly creative, so I called this
command xmapreduce. Here' I'll briefly describe
map/reduce, show the problem with it, and explain the solution
implemented in Zarkov.

Zarkov is a Web Service

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In my first post, I discussed Zarkov as an event logger. Next, I explored Zarkov as a map-reduce framework. After that, I talked about using Zarkov's aggregation definitions. Today, I'll focus on how you can get those aggregates out of Zarkov using Zarkov's built-in JSON web service.

Gevent, ZeroMQ, WebSockets, and Flot FTW!

As part of the work I've been doing on Zarkov I've had the opportunity to play around with a lot of cool technologies, among which are gevent, ZeroMQ, WebSockets, and Flot. It took a while to get to the point where I could actually get things done, but once I was there, things were incredibly simple. In this post I'll show you how to use these three techologies together to build a simple web app with realtime server push data.

Zarkov is an aggregation service

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In my first post, I discussed Zarkov as an event logger. Next, I explored Zarkov as a map-reduce framework. Today, I'll focus on Zarkov as an aggregation service suitable for doing near real-time analytics.

Zarkov is a Lightweight Map-Reduce Framework

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In my previous post, I discussed Zarkov as an event logger. While this may be useful (say for logging to a central location from several different servers), there's a bit more to Zarkov. Today I'll focus on the map-reduce framework provided by Zarkov. If you want instructions on setting up Zarkov or getting events into it, please see my previous post.

Zarkov is an event logger

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In the next few posts, I'll be going over each of the components of Zarkov and how they work together. Today, I'll focus on Zarkov as an event logger.

MongoDC: Rapid, Scalable Web Development with MongoDB, Ming, and Python

Here are the slides from my #mongodc talk for those interested....

Rapid, Scalable Web Development with MongoDB, Ming, and Python

View more presentations from Rick Copeland

MongoSF Slides

Here are the slides from my talk at MongoSF about Allura and how we're using MongoDB to build the next generation SourceForge developer tools. Enjoy!

Allura - an Open Source MongoDB Based Document Oriented SourceForge

View more presentations from Rick Copeland

Allura Sprint-orials

So with our initial announcement at PyCon and the sprints afterwards, the newly open source SourceForge project hosting platform Allura has gotten a bit of attention. One thing we want to do at SourceForge is to be responsive and open to the community, to make Allura much more than a "throw it over the wall" open source project. To that end, we've been looking at holding a sprint-orial series.

Allura, the Open Source Forge

So in kind of a soft launch, SourceForge released the project I've been working on (Allura) under the Apache license last month. The project happens to be the actual software underlying the SourceForge.net 'beta' tools, and we're hoping to get lots of community involvement, starting at PyCon.

MongoATL Video

Here's the video from my talk at MongoATL a few weeks ago. Enjoy!

http://www.10gen.com/video/mongoatlanta2011/sourceforge

MongoATL Slides

Here are the slides from my MongoATL presentation for those who might be interested.

MongoATL: How Sourceforge is Using MongoDB

View more presentations from Rick Copeland.

MongoATL!

Quick note to let anyone in the ATL area know that I'll be speaking on how SourceForge is using MongoDB at MongoATL on February 8th (next Tuesday). The registration cost is only $100 ($30 if you're a student), and it's a great way to learn about one of the most popular NoSQL databases on the scene today.