Monday, December 07, 2009
Sunday, May 03, 2009
In my ever-expanding quest to, as @jgustak recently tweeted, "introduce evil to Python to prevent even scarier evil," I have released MetaPython 0.2.2 Once again, if you aren't familiar with MetaPython, a good place to start is the tutorialwhich walks you through the construction of a macro-ized collections.namedtuple from the Python 2.6 standard library. If you already know MetaPython, here's the stuff that's new in 0.2.2.
Friday, April 17, 2009
collections.namedtuplefrom the Python 2.6 standard library. If you are already familiar with MetaPython, I will try to summarize the changes and ideas in this release here.
MetaPython 0.1 was mainly a proof of concept implementation to show that it was possible to generate a moderately useful macro facility in a short period of time. I did, however, make one really questionable decision: to use Jinja2 as the templating language for code quotes.
Don't get me wrong; Jinja2 is an awesome templating language. But if I am generating a language extension (MetaPython) that allows you to change the text of a module as just before it gets imported, wouldn't it be nice to also be able to change the text of a code quote using the same syntax? Hence version 0.2.
There was also the issue of ugly syntax with the
$are still there, I have tried to normalize their use a bit.
$always introduces a construct that should be executed or evaluated "earlier" than the surrounding code. (Inside a
defcode...:block, this means at block construction time, otherwise it means at import time.) The
?operator now has a much more limited use as an inline code quoting operator.
Other than syntax changes, one of the main things you'll notice in MetaPython 0.2 is the introduction of import-time control statements (
$if..., etc.) These allow the conditional or repeated expansion of code blocks. These constructs basically obviate the need for another template language (Jinja2) inside
Another big change and move toward normalization is that macro calls are now just import-time function calls. This means that their arguments are evaluated before they are called, not passed through as code objects. In order to get the old behavior, you can simply quote the arguments you wish to be sent through unevaluated. For instance, in the old syntax, creating a named tuple was accomplished via
?namedtuple(Point, x, y), whereas in the new syntax, you would type
$namedtuple(?Point, ?x, ?y). This makes it substantially easier to write macros which need non-code arguments, and follows the Python Zen of "Explicit is better than implicit."
There have also been significant reworking of the internal MetaPython parser and code construction machinery, although this should not be user-visible. So try it out, let me know what you think, and have fun!
Friday, April 10, 2009
Thursday, March 19, 2009
So what's all the excitement about? MetaPython introduces some hooks to allow you to modify module code just before it is seen by the Python interpreter (at what I'm calling "import time"). The import-time syntax is pretty simple, and is (almost) all denoted by a question mark
?prefix (question marks are currently syntax errors in regular Python). Here is a trivial example that defines an import-time function (which as we will see can be used as a macro) that will conditionally remove a function call (and the evaluation of its associated arguments). Suppose the following text is saved in a file "cremove.mpy":
def cremove(debug, expr):
debug = eval(str(debug))
result = expr
result = empty_result
The idea here is that
cremovewill be called with two
cremovewill convert debug to its Python code representation by calling
str()and then evaluate the result. If
debugis true, then
exprwill be returned. Otherwise a
passstatement will be returned (defined using the MetaPython import-time construct
defcodewhich defines a code template). To actually call cremove as a macro, we will need to define another MetaPython module, say "test_cremove.mpy":
log = logging.getLogger(__name__)
?from cremove import cremove
?cremove(True, log.debug('This statement will be logged'))
?cremove(False, log.debug('This statement will not be logged'))
Here, we do an import-time import (the
?from... importbusiness). This makes the module we're importing available at import-time (a regular import would be seen just as another line of Python code at import-time). To actually call cremove as a macro, we just need to prefix it with the "?" as shown.
Now, to actually test this, we'll need to install MetaPython and fire up an interpreter. MetaPython is available from the CheeseShop, so to get it just run
easy_install MetaPython. Once it's installed, we can test our MetaPython code as follows:
>>> import metapython; metapython.install_import_hook()
>>> import test_cremove
DEBUG:test_cremove:This statement will be logged
Since macro expansion can get pretty complex and it's always tricky debugging code you've never seen, the fully-expanded module is available as the
__expanded__attribute of the MetaPython module:
>>> print test_cremove.__expanded__
logging .basicConfig (level =logging .DEBUG )
log =logging .getLogger (__name__ )
def do_test ():
log .debug ('This statement will be logged')
There's a lot more to MetaPython, but hopefully this has whetted your appetite. There's a short tutorial available that shows how you can implement the
collections.namedtupleclass factory using a macro. It also shows how you can use Jinja2 syntax along with the defcode construct to dynamically produce Python code. Have fun, and let me know what you think!
Thursday, March 12, 2009
As I sat down to actually start figuring out what macro definitions and uses should look like in Python, I thought, hey, I'll just throw together a use case. But I haven't been able to come up with one (yet).
Most of the examples I found on the web focused on "hey, you can implement a 'while' loop with macros in Lisp!" or "hey, look at all the cool stuff the 'setf' macro can do!" So I started to wonder whether maybe Lisp people love macros because it allows them to extend Lisp's minimalist syntax with new constructs (like object-oriented programming with CLOS, while loops, etc.) Python, OTOH, has pretty rich syntax. It has a nice OOP system with syntactic support, while and for loops, generators, iterators, context managers, primitive coroutines, comprehensions, destructuring bind,.... -- What would I use macros for? (OK, depending on the syntax, I could add a "switch" statement, but that hardly seems worth the trouble.)
I should mention that I also saw some examples of people using macros for performance; you basically get rid of a function call and you can potentially make the inner loop of some critical function run really fast. But if that's all it buys me in Python-land (well, that and a switch statement), my motivation is pretty low. Because let's face it -- if your critical inner loop is written in pure Python, you can pretty easily throw it at Cython and get better performance than Python macros could ever provide.
So here's the question: does anyone out there have an idea of what macros would add to Python's power or expressiveness? Or maybe some Lisp, Meta OCAML, or Template Haskell hackers who can enlighten me as to what macros can add to a language with already rich syntax?
I have implemented MetaPython 0.1, a macro and code quoting system for Python, covered in the next blog post.