Friday, April 17, 2009

MetaPython 0.2 Release

For those intrepid souls who are interested in generating Python code from the macros and code quoting facilities of MetaPython, I have spun a new release. If you aren't familiar with MetaPython, a good place to start is the tutorial, which walks you through the construction of a macro for generating a macro-ized version of collections.namedtuple from the Python 2.6 standard library. If you are already familiar with MetaPython, I will try to summarize the changes and ideas in this release here.

MetaPython 0.1 was mainly a proof of concept implementation to show that it was possible to generate a moderately useful macro facility in a short period of time. I did, however, make one really questionable decision: to use Jinja2 as the templating language for code quotes.

Don't get me wrong; Jinja2 is an awesome templating language. But if I am generating a language extension (MetaPython) that allows you to change the text of a module as just before it gets imported, wouldn't it be nice to also be able to change the text of a code quote using the same syntax? Hence version 0.2.

There was also the issue of ugly syntax with the ?, $, and {%...%} operators. Although ? and $ are still there, I have tried to normalize their use a bit. $ always introduces a construct that should be executed or evaluated "earlier" than the surrounding code. (Inside a defcode...: block, this means at block construction time, otherwise it means at import time.) The ? operator now has a much more limited use as an inline code quoting operator.

Other than syntax changes, one of the main things you'll notice in MetaPython 0.2 is the introduction of import-time control statements ($for..., $if..., etc.) These allow the conditional or repeated expansion of code blocks. These constructs basically obviate the need for another template language (Jinja2) inside defcode... blocks.

Another big change and move toward normalization is that macro calls are now just import-time function calls. This means that their arguments are evaluated before they are called, not passed through as code objects. In order to get the old behavior, you can simply quote the arguments you wish to be sent through unevaluated. For instance, in the old syntax, creating a named tuple was accomplished via ?namedtuple(Point, x, y), whereas in the new syntax, you would type $namedtuple(?Point, ?x, ?y). This makes it substantially easier to write macros which need non-code arguments, and follows the Python Zen of "Explicit is better than implicit."

There have also been significant reworking of the internal MetaPython parser and code construction machinery, although this should not be user-visible. So try it out, let me know what you think, and have fun!


  1. Anonymous10:54 PM

    This looks very interesting. How do the macros interact with the debugger?

  2. @fivebells: I haven't actually done a lot of experimentation with the debugger, so my guess is "not very well." Macro debugging is, in general, difficult since the code at runtime is not the same code you originally wrote. To aid in debugging somewhat, a MetaPython module has a special attribute __expanded__ that represents the Python code that all the macros in the source expand out to.

    One debugging trick that may be useful to add is the ability to dump out a standard .py file from a .mpy file and use that for debugging. Then at least you're dealing with "regular" Python instead of a weird exec'd string of Python code.

  3. Anonymous11:18 AM

    Thanks for the information, Rick. Does MetaPython have that dumping capability?

  4. @fivebells: Not yet -- any chance I could get you to write a feature request at ?