Friday, July 18, 2008

RESTfulness in TurboGears

Mark Ramm and I were talking about how to differentiate GET, POST, PUT, and DELETE in TurboGears 2 and we came up with a syntax that's pretty cool. That's not the reason for this post, though. This morning, I noticed that our syntax is completely compatible with TurboGears 1.x -- so here's how you do it.

What we wanted was to expose a RESTful method like this:

class Root(controllers.RootController):

class index(RestMethod):
@expose('json')
def get(self, **kw):
return dict(method='GET', args=kw)
@expose('json')
def post(self, **kw):
return dict(method='POST', args=kw)
@expose('json')
def put(self, **kw):
return dict(method='PUT', args=kw)
# NOT exposed, for some reason
def delete(self, **kw):
return dict(method='DELETE', args=kw)

The TurboGears 1 implementation relies on the way that CherryPy determines whether a given property is a valid URL controller. It basically looks at the property and checks to make sure that:

  • it's callable, and

  • it has a property exposed which is true (or
    "truthy")


The "a-ha!" moment came when realizing that:

  • Classes are callable in Python (calling a class == instantiating an object)

  • Classes can have exposed attributes


So with appropriate trickery behind the scenes, the above syntax should work as-is. So here's the "appropriate trickery behind the scenes":

class RestMeta(type):
def __new__(meta,name,bases,dct):
cls = type.__new__(meta, name, bases, dct)
allowed_methods = cls.allowed_methods = {}
for name, value in dct.items():
if callable(value) and getattr(value, 'exposed', False):
allowed_methods[name] = value
return cls

The first thing I wanted to do was create a metaclass to use for RestMethod so that I could save the allowed HTTP methods. Nothing too complicated here.

import cherrypy as cp
class ExposedDescriptor(object):
def __get__(self, obj, cls=None):
if cls is None: cls = obj
allowed_methods = cls.allowed_methods
cp_methodname = cp.request.method
methodname = cp_methodname.lower()
if methodname not in allowed_methods:
raise cp.HTTPError(405, '%s not allowed on %s' % (
cp_methodname, cp.request.browser_url))
return True

This next thing is tricky. If you don't understand what a "descriptor" is, I suggest the very nice description here. The basic thing I get here is the ability to intercept a reference to a class attribute the same way the property() builtin intercepts references to object attributes.

The idea here is to use this descriptor as the exposed attribute on the RestMethod class. When CherryPy tries to figure out if the method is exposed, it calls ExposedDescriptor.__get__ and uses the result as the value of exposed. If the HTTP method in question is not exposed, then the code raises a nice HTTP 405 error, which is the correct response to sending, say, a POST to a method expecting only GETs.

The final part of the solution, the actual RestMethod, is actually pretty simple:

class RestMethod(object):
__metaclass__ = RestMeta

exposed = ExposedDescriptor()

def __init__(self, *l, **kw):
methodname = cp.request.method.lower()
method = self.allowed_methods[methodname]
self.result = method(self, *l, **kw)

def __iter__(self):
return iter(self.result)

The sequence of things is now this:

  • CherryPy traverses along to the root.index class and looks up root.index.exposed

  • root.index.exposed is intercepted by the descriptor which checks the CherryPy request method to see if it's valid for this controller, and if it is, returns True

  • CherryPy says, "Great! root.index is exposed. So now I'll call root.index(...)." This calls index's constructor, which in turn calls the appropriate method and saves the result in self.result

  • CherryPy says "Cool! root.index returned me an iterable object. I'll iterate over it to get the text to send back to the browser." This calls root.index(...).__iter__, which is just delegated to the result that the real controller gave.


At then end, we get a fully REST compliant controller with a nice (I think) syntax.

Update 2008-07-21: After some more thinking, I realized that the metaclass isn't necessary. The descriptor is more important, but also not strictly necessary. My original design did everything in the constructor of RestMethod, but this runs after all of TurboGears' validation and identity checking happens, so it's pretty inefficient. If you want the implementation with the descriptor but without the metaclass, you can do this:

import cherrypy as cp
class ExposedDescriptor(object):
def __get__(self, obj, cls=None):
if cls is None: cls = obj
cp_methodname = cp.request.method
methodname = cp_methodname.lower()
method = getattr(cls, methodname, None)
if callable(method) and getattr(method, 'exposed', False):
return True
raise cp.HTTPError(405, '%s not allowed on %s' % (
cp_methodname, cp.request.browser_url))

The benefit to using a metaclass is that the check for the existence, "callability", and exposed-ness of the method happens up front rather than on each request. Also, I don't like the fact that the metaclass pollutes the class namespace with the "allowed_methods" attribute. There's probably a way to clean that up and put the descriptor in a closure, but I haven't had time to look at it. Maybe that will be a future post....

Friday, July 11, 2008

Cascade Rules in SQLAlchemy

Last night at the PyAtl meeting, there was a question about how you define your cascade rules in SQLAlchemy mappers. I'll confess that it confused me at first, too, but here's all you need to know:

What's "cascading" in the mapper is session-based operations. This includes putting an object into the session (saving it), deleting an object from the session, etc. Generally, you don't care about all that stuff, because it Just Works most of the time, as long as you specify cascade="all" on your relation() properties in your mappers. What this means is "whatever session operation you do to the mapped class, do it to the related class as well".

One little confusing thing is that there's another thing you'll often want to specify in your cascade rules, and that's the "delete-orphan". In fact, most of my 1:N relation()s look like:

mapper(ParentClass, parent, properties=dict(
children=relation(ChildClass, backref='parent',
cascade='all,delete-orphan')
)
)

The "delete-orphan" specifies that if you ever have a ChildClass instance that is "orphaned", that is, not connected to some ParentClass, go ahead and delete that ChildClass. You want to specify this whenever you don't want ChildClass instances hanging out with null ParentClass references. Note that even if you don't specify "delete-orphan", deletes on the ParentClass instance will still cascade to related ChildClass instances. An example is probably best. Say you have the following schema and mapper setup:

photo = Table(
'photo', metadata,
Column('id', Integer, primary_key=True))
tag = Table(
'tag', metadata,
Column('id', Integer, primary_key=True),
Column('photo_id', None, ForeignKey('photo.id')),
Column('tag', String(80)))

class Photo(object): pass

class Tag(object): pass

session.mapper(Photo, photo, properties=dict(
tags=relation(Tag, backref='photo', cascade="all"),
session.mapper(Tag, tag)

I'll go ahead and create some photos and tags:

p1 = Photo(tags=[
Tag(tag='foo'),
Tag(tag='bar'),
Tag(tag='baz') ])
p2 = Photo(tags=[
Tag(tag='foo'),
Tag(tag='bar'),
Tag(tag='baz') ])
session.flush()
session.clear()

Now if I delete one of the photos, I'll delete the tags associated
with it, as well:

>>> for t in Tag.query():
... print t.id, t.photo_id, t.tag
...
1 1 foo
2 1 bar
3 1 baz
4 2 foo
5 2 bar
6 2 baz
>>> session.delete(Photo.query.get(1))
>>> session.flush()
>>> for t in Tag.query():
... print t.id, t.photo_id, t.tag
...
4 2 foo
5 2 bar
6 2 baz

At this point, everything is the same whether I specify
"delete-orphan" or not. The difference is in what happens when I
just remove an item from a photo's "tags" collection:

>>> p2 = Photo.query.get(2)
>>> del p2.tags[0]
>>> session.flush()
>>> for t in Tag.query():
... print t.id, t.photo_id, t.tag
...
4 None foo
5 2 bar
6 2 baz

See how the "foo" tag is just hanging out there with no photo?
That's what "delete-orphan" is designed to prevent. If we'd
specified "delete-orphan", we'd have the following result:

>>> p2 = Photo.query.get(2)
>>> del p2.tags[0]
>>> session.flush()
>>> for t in Tag.query():
... print t.id, t.photo_id, t.tag
...
5 2 bar
6 2 baz

So there you go. If you don't mind orphans, then use
cascade="all" and leave off the
"delete-orphan". If you'd rather have them disappear when
disconnected from their parent, use
cascade="all,delete-orphan".

PyAtl: SQLAlchemy Theme Night

Well, last night was the Python Atlanta user group meeting (PyAtl). It had been a while since I've been, and I'd forgotten how fun it can be. The theme was SQLAlchemy, and the speaker lineup was me, Brandon Craig Rhodes, and James Fowler.

The meeting started off with "shooting the breeze" as usual, and then moved into my presentation "Essential SQLAlchemy", which gives a 30 minute overview of the basics of SQLAlchemy. Here are the usual links to slides and the video:



After my talk, Brandon Craig Rhodes (who is, by the way, an incredibly lively presenter, using nothing but emacs!) gave a talk "SQLAlchemy Advanced Mappings" that focused on using the ORM layer in SQLAlchemy. It really was more of a mini-tutorial that took you through basic mappings all the way through relations, backrefs, and more. SQLAlchemy is an amazingly rich library, and it's hard to squeeze a talk into half an hour. Here's the video:



After Brandon, James Fowler did a "now for something completely different" kind of talk on wxPython, "WxPython Quick Bite", focusing on how you can make wxPython (designed to be event-driven and single-threaded) play nicely in a multi-threaded environment. Unfortunately the start of the video was cut off as I feverishly tried to download the other two videos to make room for James's talk. I'll post the video as soon as it gets uploaded.

I'd be remiss if I didn't thank O'Reilly for "sponsoring" the meetup with a giveaway of a number of books (including 9 copies of Essential SQLAlchemy, which I stuck around afterwards to sign). We also had a couple of copies of Beautiful Code, Beginning Development with Python Gaming, and Hackerteen to give away. A great time was had by all!