Monday, February 27, 2006

Django-like automated admin interface

Categories: , , ,

As I continue to upgrade ConsulTracker and work on my new (not-yet announced) project, I often find myself wanting a nice CRUD-like interface, something akin to what I’ve heard Django has. While less masochistic developers might go for the “real deal” of Django, I chose to begin adding admin-style CRUD pages for TurboGears, the Python web framework with which I’m most familiar.





Requirements



Now, TurboGears already has something called “FastData”, but I’ve found it somewhat limiting. What I started with, then, is a set of requirements:



Table View



There should be a (HTML) table view of the (SQL) table which shows all rows of the (SQL) table.




  • The columns shown for each table must be customizable with a minimum of custom code.

  • When a column contains a foreign key, something “useful” must show up in that column (e.g. not the numeric “id” column of the foreign table)

  • The table should have the capability of displaying columns from foreign-key’d tables as native columns. (This is kind of like creating an SQL view containing columns from multiple tables.)

  • The table should have a link for editing each row, a link for marking a row for deletion, and a button for actually deleting the marked items. This is really a pet peeve of mine. GET-style links should not modify data in a well-designed web application. That can lead to all sorts of nastiness, especially when dealing with web spiders that follow all links. So I’ll keep all the data-modifying stuff in buttons.



From these requirements, I determined that I needed some kind of information, call it “CRUD metadata” or crudmeta for short, which contained the following information for each SQLObject model I want to view:




  • The fields to include in the table (note that these can be sub-fields, as in “address.street1”, etc.)

  • The “name” of each row in the table—this is for displaying the row as a “foreign key” in another table.



Edit View



There should be an “edit” view which allows for the following functionality:




  • Each SQLObject property refrenced in “crudmeta” should have a place on the form.

  • The fields in the form should make sense for their data type. This means:

    • TextFields for StringCol, IntCol, FloatCol, etc. (with data validation)

    • CalendarDate[Time]Picker for Date[Time]Col

    • Select fields for ForeignKey – populated with the “name” from crudmeta, of course!

    • Tables for MultiJoin fields with the similar functionality as the “Table View” above.



  • AJAX-y goodness to allow you to “drill down” in the object hierarchy without losing your context.



Note that some (most) of these fields have TurboGears 0.9 widgets-style names. Not a coincidince. I want to reuse as much as possible from existing TurboGears development so as to avoid “re-inventing the wheel.”



Status & next steps



As of today (2006–02-27), the basic functionality is there. It’s not pretty, it doesn’t handle errors well, it needs refactoring badly, and it mixes model and view a bit from MVC, but it exists. My current plan is to clean it up a bit this week and (hopefully) make it available by next weekend. To whet your appetite, here is an (abbreviated) model from my new application with the CRUD annotations present. (Currently, this is all the custom code you need in order to create the above-described interface.)



class Store(SQLObject):
class crudmeta:
fields=crud.fields(‘name’, ‘note’,
‘address.street1’, ‘address.street2’,
‘address.city’, ‘address.state’, ‘address.zip’,
(‘distributionCenter’, ‘Distribution Center’),
‘orders’)
key=‘name’

name=StringCol(default=uniqueValue(‘Store’, ‘name’, ‘store’),
alternateID=True)
note=StringCol(default=’’)
distributionCenter=ForeignKey(‘DistributionCenter’,default=None,
cascade=None)
address=ForeignKey(‘Address’,default=newAddressID)
orders=MultipleJoin(‘StoreOrder’)
deliveries=MultipleJoin(‘Delivery’)




Friday, February 17, 2006

Very Simple WSGI Overview

Categories: , , ,
In all the posting and hype about full-featured frameworks, you may have overlooked a very small "un-framework", the Python web server gateway interface (WSGI). It's generally an option for deploying the large frameworks such as TurboGears or Django. What follows is a very simple and brief overview of how you can create a WSGI-compliant application server.

First off, the WSGI specification itself is a decent read, and I'd be amiss if I didn't at least mention it. Now, on to the simple overview!

Overview


First off, you need to realize that WSGI is exactly what its name implies: an interface. The best way I've found to think of it is "CGI for Python." In CGI, the shell is invoked to run some script. The shell's environment is populated with values from the HTTP request, and the script's output is returned to the client. WSGI is similar, substituting a Python function for the script, a Python dict for the shell environment, and skipping the shell altogether. A basic WSGI application server has the following outline:

def MyApplication(environ, start_response):
try:
....maybe do some stuff in response to the environ arg...
write_fn = start_response('200 OK', [('Content-type', 'text/html')...]) # send headers
....maybe do some more stuff....
... EITHER ...
yield some things
.... OR ....
return
.... OR ....
write_fn(response_text) # deprecated
except:
start_response('500 OOPS', [('Content-type', 'text/html')...], sys.exc_info)
... yield, write, or return the text of the error page ...

Your application server, then, is just a function (or other callable) that takes two arguments, an "environment" and a "start_response". In the recommended implementation, your server will either return an iterable (generally a list of strings) or itself be an iterable (generally, a generator). The minimal "hello, world!" application is below:

def MyApplication(environ, start_response):
start_response('200 OK', [('Content-type', 'text/plain')])
yield "Hello, world!"

The "environment" is just a dict of strings, much like the CGI environment. The values available are summarized below. The "start_response" is a callable that your server must call to send the HTTP Headers. You can call it up to twice, once for "normal" headers, and once for "error" headers. If you call it a second time, you must call it before generating any output, and you must call it with an "exc_info" object. The original headers (if there were any) will be overwritten by the new headers.

To do anything useful, you'll need to parse two main variables in "environ": "PATH_INFO" and "QUERY_STRING". "PATH_INFO" gives you the "rest of the path" after the mount point for your application server, and "QUERY_STRING" gives you - you guessed it - the query string. You can then implement whatever kind of URL->object mapping your heart desires, whether it be CherryPy-style object publishing, or Django-style regular expressions. You could use the functions in Python's standard cgi module to parse the query string, but Ian Bicking has a great tutorial on how to use Paste to simplify matters quite a bit. All the other WSGI variables that are available in the environment are documented below.

Environment


The variables available in the environ dict are summarized below. For the examples, assume the user requested (using GET) "http://server.com/some/path/myserver/more/path?query_args", and that the application server was mounted at "http://server.com/some/path/myserver".



















VariableExampleDescriptionAlways Present?
REQUEST_METHOD"GET"HTTP method, generally GET or POSTYes
SCRIPT_NAME"/some/path/myserver"Location in URL of application serverNo - if application server is mounted at server root
PATH_INFO"/more/path"The rest of the path after the application rootNo - for instance, if user requests "http://server.com/some/path/myserver"
QUERY_STRING"query_args"Anything after the "?" in the URLNo
CONTENT_TYPE<absent>Any Content-Type fields in the HTTP requestNo
CONTENT_LENGTH<absent>Any Content-Length fields in the HTTP requestNo
SERVER_NAME"server.com"The server name part of the URLYes
SERVER_PORT"80"The server port part of the URLYes
SERVER_PROTOCOL"HTTP/1.1"The request HTTP protocolYes
HTTP_*<absent>Other HTTP headers in requestNo
wsgi.versionThe tuple (1,0)WSGI version IDYes
wsgi.url_scheme"http"The initial part of the URLYes
wsgi.input<empty file-like object>An object from which the request body can be read - very useful for POSTsYes
wsgi.errors<file-like object>A file-like object to which the application server can write text errors to be logged by the web serverYes
wsgi.multithreadFalseWhether the application may be simultaneously invoked in a multithreaded mannerYes
wsgi.multiprocessTrueWhether the application may be simultaneously invoked in a multiprocess mannerYes
wsgi.run_onceFalseTune the application to expect to only run once (e.g. turn off caching)Yes


Thursday, February 16, 2006

Effective Decorators

Michele Simionato has written a very nice-looking module that allows decorators to be used quite effectively:

The aim of the decorator module it to simplify the usage of decorators for the average programmer, and to popularize decorators usage giving examples of useful decorators, such as memoize, tracing, redirecting_stdout, locked, etc.

Even if you don't end up using her module, I think the documentation is a great read, and a good example of how to use your own decorators more effectively.

Tuesday, February 14, 2006

Topics

Now, I'm a guy who can go on and on about a technical topic ad nauseum. (Ask my wife!) But sometimes it takes a while for me to get started. So if there are any topics you'd like me to cover here, particularly regarding Python, please post it in the comments. I'd like to make this blog a useful resource, but my creativity-challenged brain may need a bit of help. Thanks!