Extending Zarkov's Map/Reduce

Since I've been working on Zarkov, I've been writing a few
map/reduce jobs. One of the things I noticed about map/reduce, at
least as I've implemented it in Zarkov, is that it's pretty
inefficient if you want t o generate several aggregate views of the
same data (like say an event stream). In order to meet Zarkov's
performance requirements without making my operations team
too angry, I decided to "extend" map/reduce, as it's
implemented in Zarkov. I'm not terribly creative, so I called this
command xmapreduce. Here' I'll briefly describe
map/reduce, show the problem with it, and explain the solution
implemented in Zarkov.

Zarkov is a Web Service

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In my first post, I discussed Zarkov as an event logger. Next, I explored Zarkov as a map-reduce framework. After that, I talked about using Zarkov's aggregation definitions. Today, I'll focus on how you can get those aggregates out of Zarkov using Zarkov's built-in JSON web service.

Gevent, ZeroMQ, WebSockets, and Flot FTW!

As part of the work I've been doing on Zarkov I've had the opportunity to play around with a lot of cool technologies, among which are gevent, ZeroMQ, WebSockets, and Flot. It took a while to get to the point where I could actually get things done, but once I was there, things were incredibly simple. In this post I'll show you how to use these three techologies together to build a simple web app with realtime server push data.

Zarkov is an aggregation service

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In my first post, I discussed Zarkov as an event logger. Next, I explored Zarkov as a map-reduce framework. Today, I'll focus on Zarkov as an aggregation service suitable for doing near real-time analytics.

Zarkov is a Lightweight Map-Reduce Framework

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In my previous post, I discussed Zarkov as an event logger. While this may be useful (say for logging to a central location from several different servers), there's a bit more to Zarkov. Today I'll focus on the map-reduce framework provided by Zarkov. If you want instructions on setting up Zarkov or getting events into it, please see my previous post.

Zarkov is an event logger

Over the past few weeks I've been working on a service in Python that I'm calling, in the tradition of naming projects after characters in Flash Gordon, Zarkov. So what exactly is Zarkov? Well, Zarkov is many things (and may grow to more):

• Zarkov is an event logger
• Zarkov is a lightweight map-reduce framework
• Zarkov is an aggregation service
• Zarkov is a webservice

In the next few posts, I'll be going over each of the components of Zarkov and how they work together. Today, I'll focus on Zarkov as an event logger.