Tuesday, September 25, 2012

MongoDB Schema Design at Scale

I had the recent opportunity to present a talk at MongoDB Seattle on Schema Design at Scale. It's basically a short case study on what steps the MongoDB Monitoring Service (MMS) folks did to evolve their schema, along with some quantitative performance comparisons between different schemas. Given that one of my most widely read blog posts is still MongoDB's Write Lock, I decided that my blog readers would also be interested in the quantitative comparison as well.

Thursday, September 13, 2012

Python and MongoDB Travel Plans

It's been a little longer than normal between articles, so I wanted to let everyone know what I've been up to lately, and hope that I'll be able to see some of you at some upcoming events. So here's a summary of the things I'll be involved with over the next couple of months:

MongoDB Seattle

I have the privilege of speaking at MongoDB Seattle tomorrow, September 14, 2012. My talk is on "Schema Design at Scale", and wll have some insights based on experience 10gen gained when developing the MongoDB Monitoring Service. I'll even have some cool graphs based on experiments I did to illustrate the schema design tips I'll be giving.

MongoDB for Developers Online Training

I'm offering an 4-week online training class starting this Monday at 8pm EDT. I'll be delivering the training in 2 parts each week: a "lecture" portion on Mondays and an unstructured "office hours" on Tuesdays where I'll be available for questions. If you're interested, you can sign up by clicking the following link: Register for MongoDB for Developers. Right now the class is quite small, so I'll be able to spend a lot of time answering questions and interacting with students.

Python Training in Beijing

I'll be in Beijing, China to deliver an onsite "Fast Track to Python" class from October 5-10. The actual training is only on the 8-10, so if anyone's interested in getting together before that, I'd love to hang out in the People's Republic.

Big Dive and MongoTorino

I'm also happy to announce that I'm one of the featured instructors for the Big Dive training event in Turin, Italy this October. I'll be teaching a 2-day class on MongoDB and a 1-day class on Gevent, so if you're in northern Italy, I'd love to see you there! I'll also be at MongoTorino on October 19th, but the webpage for that doesn't seem to be updated yet. I'll be in Turin from October 13-20 if you want to come to either event, or just hang out over a bottle of wine.

PyCarolinas

To cap off my world tour I'm happy to be able to speak on Gevent and Socket.io at PyCarolinas on October 21st, so I'd love to see any of you there. Sadly I'll have to miss the 1st day of PyCarolinas on my way back from Italy.

And that's about it (and that's enough!). I'd love to see any of you who are in any of the cities mentioned above when I'm passing through, so feel free to contact me on twitter (@rick446), email (rick at arborian.com), or in the comments below!

Tuesday, August 28, 2012

Using ZeroMQ devices to support complex network topologies

Continuing in my ZeroMQ series, today I'd like to look at ZeroMQ "devices" and how you can integrate ZeroMQ with Gevent so you can combine the easy networking topologies of ZeroMQ with the cooperative multitasking of ZeroMQ. If you're just getting started with ZeroMQ, you might want to check out the following articles:

And if you want some background on Gevent, you might want to check out that series at the following links:

Once you're caught up, let's get started...

Tuesday, August 21, 2012

ZeroMQ flow control and other options

In a previous post, I provided an introduction to ZeroMQ. Continuing along with ZeroMQ, today I'd like to take a look at how you manage various "socket options" in ZeroMQ, particularly when it comes to flow control. If you've never used ZeroMQ, I recommend reading my previous post first. Once you're caught up, let's get started...

Monday, August 20, 2012

Python and MongoDB Online Training

One thing that I've been planning on offering as part of Arborian's services is online training. For the inaugural class, I've decided to offer a MongoDB and Python developer training class. Read on to hear more about this class...

Tuesday, August 14, 2012

Distributed Systems with ZeroMQ

Departing a bit from my current series on gevent and Python, today I want to take a look at a different networking technology that's been gaining traction: ZeroMQ. So without further ado, let's jump right in...

Wednesday, August 08, 2012

Building Web Applications with Gevent's WSGI Server

Continuing on in my series on gevent and Python, this article discusses how to use gevent to power your Python WSGI web applications. If you're just getting started with Gevent, you might want to read the previous articles in this series first:

And now that you're all caught up, let's jump into gevent's WSGI support...

Tuesday, August 07, 2012

Building TCP Servers with Gevent

Continuing on in my series on gevent and Python, this article discusses how to build TCP servers using the infrastructure provided with gevent. If you're just getting started with Gevent, you might want to read the previous articles in this series first:

And now that you're all caught up, let's jump into gevent servers...

Thursday, August 02, 2012

Greening the Python standard library with gevent

Continuing on in my series on gevent and Python, this article deals with what you need to do when want to use the Python standard library with gevent, showing how gevent provides a way to monkey-patch the standard library to make it compatible with gevent. If you're just getting started with Gevent, you might want to read the previous articles in this series first:

And now that you're all caught up, let's get started with gevent...

Tuesday, July 31, 2012

Last Chance for Early Bird Atlanta MongoDB Training

As announced earlier, I'm offering some MongoDB training classes in the first week of August in Atlanta. So why am I cluttering up your inbox today? Mainly because the early bird price of $400 per day ends at 11:59 pm (EDT) tonight, after which the price goes up to $500.

Monday, July 30, 2012

Gevent and Greenlets

Continuing on in my series on gevent and Python, this article gets down into practical details, showing you how to install gevent and get started with basic greenlet operations. If you're just getting started with Gevent, you might want to read the previous articles in this series first:

And now that you're all caught up, let's get started with gevent...

Tuesday, July 24, 2012

Two MongoDB books (one now, one early 2013)

Today I'm happy to make two book-related announcements.

MongoDB with Python and Ming Book

First off, I've collected, edited, and expanded upon the the MongoDB and Python series I've been working on at Just a Little Python and released it as an eBook. MongoDB with Python and Ming is now available on Lulu as an epub and on Amazon (at MongoDB with Python and Ming). In the book, I cover the following topics:

  • Using PyMongo, from installing & basic queries to the new aggregation framework
  • Overview of tuning your application for the best performance under MongoDB including discussions of replication and sharding
  • Using the Ming toolkit to enforce your schema in MongoDB
  • Using Ming's object-document mapper (ODM) to raise the abstraction level of your MongoDB programming
  • Various tips, tricks, and goodies with Ming including Mongo-in-Memory, extending the ODM, and schema migrations

MongoDB Applied Design Patterns

The second announcement is that I've agreed to write another book for O'Reilly, tentatively titled MongoDB Design Patterns. That one's not done yet, and it's still early in the authoring / publication process, but it should be available early 2013. Sadly I don't have a nice product page link to share with you yet, but I will leave you with a summary of the topics I plan to cover:

  • Part 1: MongoDB Design Patterns
    • Embedding versus referencing documents
    • Using polymorphic schemas
    • Using complex atomic updates
    • Optimistic updates with compensation (in lieu of multi-document transactions)
  • Part 2: Use Cases
    • Operational intelligence / real-time analytics
    • E-Commerce
    • Content management systems
    • Online advertising networks
    • Social Networking
    • Online Gaming

I hope that's enough to whet your appetite. Of course, the book's not finished, so if you have any other topics you'd love to see covered in such a book, I'd love to hear about it in the comments!

Monday, July 23, 2012

Gevent, Threads, and Benchmarks

In a previous post, I gave an introduction to gevent to show some of the benefits your application might get from using gevent greenlets instead of threads. Some people, however, took issue with my benchmark code, saying that the threaded example was contrived. In this post, I'll try to answer some of the objections.

(It actually turns out that there was a bug in the version of ab I was using to test, as well, so I re-ran the tests from the previous post, too.)

Introduction to Gevent

In a previous post, I described how to build a web chat server with socket.io and gevent. If you're trying to actually learn gevent, socket.io, however, it's probably not the best place to start. So I figured I'd write this post and provide an overview of gevent.

[Update 2012-07-24] In response to some criticisms of the micro-benchmarks in this post, I reworked the benchmarks and wrote an updated gevent and threads post. Make sure you read that one for more perspective on greenlets vs. threads.

Tuesday, July 17, 2012

MongoDB Training Classes in Atlanta

As promised earlier, we have now officially launched MongoDB and Python training classes. The first batch of classes will be offered between August 7-10, 2012, with early bird registration closing on July 31. We'd love to have you attend one or more classes, so read on for more info!

Monday, July 16, 2012

Realtime Web Chat with Socket.io and Gevent

In a previous post, I described how to build a realtime analytics graph with socket.io. That was nearly a year ago, however, and both socket.io and the Python library gevent-socketio have both undergone significant changes. In this post, then, I'll use the latest versions of each to build a web chat server.

Tuesday, July 10, 2012

Rick's Resources Blog

Since starting my own consulting business (and actually, quite a while before that), I've run across various resources, from hosting, to web apps, to actual, tangible products I've bought, that others might find interesting. In order let those who want to hear about such things hear about them, while keeping this blog more technically focused, I've created a new blog, Rick's Resources, that will cover the various things I've found interesting over the years.

Hopefully this blog will be useful to those of you who are interested, and as I mentioned, for those of you who aren't, just stick around here and we'll get back to Python momentarily. For those who are interested, my inaugural post on the resources blog is The Five Minute DreamHost Blog, where I walk you through setting up a blog on DreamHost. (It's not technically five minutes since you have to wait for some emails, but it's a reasonable facsimile thereof, I hope.)

And now, back to your regularly scheduled Pythonning....



Monday, July 09, 2012

Multi-Master Replication in MongoDB

Since starting my own consulting business, I've had the opportunity to work with lots of interesting technologies. Today I wanted to tell you about some interesting technology I got to develop for MongoDB: multi-master replication.

Wednesday, July 04, 2012

Relational Mapping with MongoDB, Ming, and Python

Continuing on in my series on MongoDB and Python, this article delves into the object-document mapper (ODM) included with the Python MongoDB toolkit Ming. If you're just getting started with MongoDB, or with Ming, you might want to read the previous articles in the series first:

And now that you're all caught up, let's take a look at what makes up an object document mapper, anyway...

Tuesday, June 19, 2012

Declarative Schemas for MongoDB in Python using Ming

Continuing on in my series on MongoDB and Python, this article continues to explore the Python MongoDB toolkit Ming, this time looking at an alternative, declarative syntax for schema definition. If you're just getting started with MongoDB, you might want to read the previous articles in the series first:

And now that you're all caught up, let's take a look at that declarative syntax....

Friday, June 08, 2012

Schema Maintenance with Ming and MongoDB

Continuing on in my series on MongoDB and Python, this article introduces the Python MongoDB toolkit Ming and what it can do to simplify your MongoDB code and ease maintenance. If you're just getting started with MongoDB, you might want to read the previous articles in the series first:

And now that you're all caught up, let's jump right in with Ming....

Tuesday, June 05, 2012

Python and MongoDB Training Classes, Anyone?

Departing just a bit from my current series on how to effectively use MongoDB from Python, today I wanted to let you, my blog readers, know that as of this past February I left SourceForge to start a new python and mongodb consulting business, Arborian Consulting. Why is this (hopefully) relevant to you? At this past PyCon, I had the opportunity to lead a python and mongodb tutorial and had a great time, but realized that the three hour format is just not enough time. So I'm hoping to start some full- or multi-day training classes, but I need your input to know what and where to offer them.

Friday, June 01, 2012

Using MongoDB's New Aggregation Framework in Python (MongoDB Aggregation Part 2)

Continuing on in my series on MongoDB and Python, this article will explore the new aggregation framework introduced in MongoDB 2.1. If you're just getting started with MongoDB, you might want to read the previous articles in the series first:

And now that you're all caught up, let's jump right in....

Wednesday, May 30, 2012

Aggregation in MongoDB (Part 1)

In some previous posts on mongodb and python, pymongo, and gridfs, I introduced the NoSQL database MongoDB how to use it from Python, and how to use it to store large (more than 16 MB) files in it. Here, I'll be showing you a few of the features that the current (2.0) version of MongoDB includes for performing aggregation. In a future post, I'll give you a peek into the new aggregation framework included in MongoDB version 2.1.

Review: Web2Py Application Development Cookbook

Changing things up a bit, today I have a special treat, as Jay Kelkar, an avid Pythonista and fellow member of the Python Atlanta user group, has agreed to write a guest post reviewing the web2py Application Development Cookbook. If you give him half a chance, Jay will be happy to expound upon the benefits of Web2Py, so this review seemed a perfect fit for him. So without further introduction, here's Jay's review (full disclosure: the Amazon links below are affiliate links):

Friday, May 25, 2012

GridFS: The MongoDB Filesystem

In some previous posts on mongodb and python and pymongo, I introduced the NoSQL database MongoDB and how you can use it from Python. This post goes beyond the basics of MongoDB and pymongo to give you a taste for MongoDB's take on filesystems, GridFS.

Tuesday, January 31, 2012

Moving along with PyMongo

In a previous post, I introduced MongoDB and its Python driver pymongo. Here, I continue in the same vein, describing in more detail how you can become productive using pymongo.

Wednesday, January 18, 2012

Getting Started with MongoDB and Python

If you've been following this blog for a while, you've seen me mention MongoDB more than once. One exciting thing for me is that I'll be co-teaching a tutorial at PyCon this year on Python and MongoDB that will cover MongoDB, PyMongo, and Ming. So to hopefully whet your appetite for learning more at the tutorial, I thought I'd write a few posts covering MongoDB, PyMongo, and Ming from a beginner's perspective.