Saturday, December 31, 2011

MongoDB's Write Lock

MongoDB, as some of you may know, has a process-wide write lock. This has caused some degree of ridicule from database purists when they discover such a primitive locking model. Now per-database and per-collection locking is on the roadmap for MongoDB, but it's not here yet. What was announced in MongoDB version 2.0 was locking-with-yield. I was curious about the performance impact of the write lock and the improvement of lock-with-yield, so I decided to do a little benchmark, MongoDB 1.8 versus MongoDB 2.0.

Monday, October 31, 2011

MongoDB Rocks My World

I am delighted to announce that I'll be speaking at several MongoDB-related events over the next couple of months. So if you're in or near Dallas, Seattle, or Silicon Valley, I'd love it if you could make it to one of my talks. 10gen (the makers of MongoDB) do a great job of putting on regional conferences that are extremely reasonably priced (typically $30 student and $50 early bird), and there's usually a nice after-party for each one, so there's no excuse not to go.

So anyway, here are the conferences I'll be speaking at:
I also wanted to say a few words on why I enjoy using MongoDB and why it is such a good fit for a lot of the problems I face...

Wednesday, August 10, 2011

MongoDB Atlanta User Group

So last night I had the opportunity to speak about Zarkov at the Atlanta MongoDB user group (not to be confused with the MongoATL conference). It was a good crowd, and we had a good time munching on pizza beforehand and talking about MongoDB afterward. Anyway, I did a little realtime demo combining some stuff I've been doing with Socket.io, Zarkov, and Flot and promised I'd make the code available. A mostly-functional version is available at http://sf.net/u/rick446/mongodb-atl/.

Also, I figured I'd use this post to collect all the presentation paraphernalia from last night, so below are the slides and somewhat poor quality video I recorded. Hope you manage to enjoy it anyway!

Sunday, August 07, 2011

WebSockets to Socket.io

Update July 16, 2012 There have been several updates to gevent-socketio and socket.io itself. For an updated intro to these libraries, please see m y new post on Realtime Web Chat with Socket.io and Gevent


In a previous blog post, I showed how you can use Gevent, ZeroMQ, WebSockets, and Flot to create a nice asynchronous server that graphs events in real time to multiple web consumers. Unfortunately, Chrome is just about the only browser that my demo worked in due to some issues with the WebSockets standard. So I've updated the example to use Socket.io, a WebSockets-like Javascript library that will work across multiple browsers.


Tuesday, July 26, 2011

Extending Zarkov's Map/Reduce

Since I've been working on Zarkov, I've been writing a few
map/reduce jobs. One of the things I noticed about map/reduce, at
least as I've implemented it in Zarkov, is that it's pretty
inefficient if you want t o generate several aggregate views of the
same data (like say an event stream). In order to meet Zarkov's
performance requirements without making my operations team
too angry, I decided to "extend" map/reduce, as it's
implemented in Zarkov. I'm not terribly creative, so I called this
command xmapreduce. Here' I'll briefly describe
map/reduce, show the problem with it, and explain the solution
implemented in Zarkov.

Monday, July 25, 2011

Zarkov is a Web Service

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

In my first post, I discussed Zarkov as an event logger. Next, I explored Zarkov as a map-reduce framework. After that, I talked about using Zarkov's aggregation definitions. Today, I'll focus on how you can get those aggregates out of Zarkov using Zarkov's built-in JSON web service.

Saturday, July 23, 2011

Gevent, ZeroMQ, WebSockets, and Flot FTW!

As part of the work I've been doing on Zarkov I've had the opportunity to play around with a lot of cool technologies, among which are gevent, ZeroMQ, WebSockets, and Flot. It took a while to get to the point where I could actually get things done, but once I was there, things were incredibly simple. In this post I'll show you how to use these three techologies together to build a simple web app with realtime server push data.

Friday, July 22, 2011

Zarkov is an aggregation service

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

In my first post, I discussed Zarkov as an event logger. Next, I explored Zarkov as a map-reduce framework. Today, I'll focus on Zarkov as an aggregation service suitable for doing near real-time analytics.

Thursday, July 21, 2011

Zarkov is a Lightweight Map-Reduce Framework

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

  • Zarkov is an event logger
  • Zarkov is a lightweight map-reduce framework
  • Zarkov is an aggregation service
  • Zarkov is a webservice

In my previous post, I discussed Zarkov as an event logger. While this may be useful (say for logging to a central location from several different servers), there's a bit more to Zarkov. Today I'll focus on the map-reduce framework provided by Zarkov. If you want instructions on setting up Zarkov or getting events into it, please see my previous post.

Wednesday, July 20, 2011

Zarkov is an event logger

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

  • Zarkov is an event logger
  • Zarkov is a lightweight map-reduce framework
  • Zarkov is an aggregation service
  • Zarkov is a webservice

In the next few posts, I'll be going over each of the components of Zarkov and how they work together. Today, I'll focus on Zarkov as an event logger.

Wednesday, May 25, 2011

MongoSF Slides

Here are the slides from my talk at MongoSF about Allura and how we're using MongoDB to build the next generation SourceForge developer tools. Enjoy!

Thursday, March 31, 2011

Allura Sprint-orials

So with our initial announcement at PyCon and the sprints afterwards, the newly open source SourceForge project hosting platform Allura has gotten a bit of attention. One thing we want to do at SourceForge is to be responsive and open to the community, to make Allura much more than a "throw it over the wall" open source project. To that end, we've been looking at holding a sprint-orial series.

Wednesday, March 09, 2011

Allura, the Open Source Forge

So in kind of a soft launch, SourceForge released the project I've been working on (Allura) under the Apache license last month. The project happens to be the actual software underlying the SourceForge.net 'beta' tools, and we're hoping to get lots of community involvement, starting at PyCon.

Tuesday, February 22, 2011

Tuesday, February 08, 2011

MongoATL Slides

Here are the slides from my MongoATL presentation for those who might be interested.

Thursday, February 03, 2011

MongoATL!


Quick note to let anyone in the ATL area know that I'll be speaking on how SourceForge is using MongoDB at MongoATL on February 8th (next Tuesday). The registration cost is only $100 ($30 if you're a student), and it's a great way to learn about one of the most popular NoSQL databases on the scene today.