As I sat down to actually start figuring out what macro definitions and uses should look like in Python, I thought, hey, I'll just throw together a use case. But I haven't been able to come up with one (yet).
Most of the examples I found on the web focused on "hey, you can implement a 'while' loop with macros in Lisp!" or "hey, look at all the cool stuff the 'setf' macro can do!" So I started to wonder whether maybe Lisp people love macros because it allows them to extend Lisp's minimalist syntax with new constructs (like object-oriented programming with CLOS, while loops, etc.) Python, OTOH, has pretty rich syntax. It has a nice OOP system with syntactic support, while and for loops, generators, iterators, context managers, primitive coroutines, comprehensions, destructuring bind,.... -- What would I use macros for? (OK, depending on the syntax, I could add a "switch" statement, but that hardly seems worth the trouble.)
I should mention that I also saw some examples of people using macros for performance; you basically get rid of a function call and you can potentially make the inner loop of some critical function run really fast. But if that's all it buys me in Python-land (well, that and a switch statement), my motivation is pretty low. Because let's face it -- if your critical inner loop is written in pure Python, you can pretty easily throw it at Cython and get better performance than Python macros could ever provide.
So here's the question: does anyone out there have an idea of what macros would add to Python's power or expressiveness? Or maybe some Lisp, Meta OCAML, or Template Haskell hackers who can enlighten me as to what macros can add to a language with already rich syntax?
Update 2008-03-19
I have implemented MetaPython 0.1, a macro and code quoting system for Python, covered in the next blog post.
I'm learning PLT Scheme right now and while the IDE is quite cool, I don't see much point to macros either.
ReplyDeleteI think macros could create idiosyncratic constructs, which not pythonic.
Macros could allow you to not use explicit self in every class method...
ReplyDelete@anonymous: Not that I think it's a good idea, but you can actually do that without macros using metaclasses.
ReplyDeleteA coworker and I were just discussing this the other day...
ReplyDeleteIn C++ development at my previous employment, we used macros all the time, to reduce repeated code patterns. The most common example was in exception handling, for example, catching, logging, and then re-throwing an exception.
As I was writing "except Exception, e: ..." for yet another time the other day, I realized that with macros I could have avoided this repetition. Avoiding repetition means I only have to write it once, and thus, less potential for bugs!
@chphilli: I kind of see what you're saying, but couldn't you achieve the same thing with either a decorator (to guard a function) or a context manager (to guard an arbitrary block of code)?
ReplyDeletewith logging_exceptions:
some
potentially
exception
raising
stuff
Yeah, I suppose you could -- I like the idea of a context manager approach -- in fact, I'll probably implement it that way sometime next week now that you point it out.
ReplyDeleteI think the original point probably still stands though -- any where you have a set of boiler-plate code, it may be nice to have macro support.
That said, as you've pointed out, it's nowhere near as important as in less flexible languages like C++!
I guess Lisp programmers when creating templating or configuration languages want to keep programming in Lisp plus a few additions, whereas Python programmers either need something completely different so that a few macros would not help them much anyway or if they want Python they want to stay as close as possible to its syntax.
ReplyDeleteMacros allow your code-definition code to look more like regular code.
ReplyDeleteThe type() example from last night's PyATL meetup is a good example:
class Test{BROWSER}(BrowserTestCase):
browser = {BROWSER}
looks a lot more like a normal Python class definition than:
type("Test%s" %(CLASSNAME, BROWSER), (BrowserTestCase,), dict(browser=BROWSER))
@Zellyn: I see what you're saying, and I agree that the type(...) approach can be off-putting, but I can also just write the following:
ReplyDeleteclass Test(BrowserTestCase):
browser = browser
Test.__name__ = 'Test%s' % browser
In Python 2.6, I could even write a class decorator to clean it up:
@setname('Test%s' % browser)
class Test(BrowserTestCase):
browser = browser
where I have
def setname(name):
def inner(o):
o.__name__ = name
return o
return inner
Another thing I didn't like about the type(...) approach is that it doesn't account for metaclasses, but that's a minor pet peeve.
Actually, since you mentioned this, I can now think of a good use case for macros -- it would clean up the collections.namedtuple class generator quite a bit. Right now, it puts together a string and eval()s it. Kind of a poor man's macro anyway.
Rick, it was good seeing you last night. I think Python and Lisp are pretty close in terms of expressiveness. They have different philosophies though, and it attributes to why/how they are different. I much prefer Python's syntax to Lisp - code is generally much more pleasant to read, which is a big win. On the other hand, I like the simplicity of Lisp, in that there's just not that much to it. Macro's are so powerful you can do almost anything with it. On the other hand in Python, you have a large set of tools at your disposal(metaclasses, decorators, generators, the *with* clause, etc.). There's more you need to learn and more to chose from. Also, since they are language features, you can't extend them yourself - we have to design them by committee. Python wasn't always this expressive - It took many years to get to this point. Lispers were already enjoying their Macros almost at the beginning. Back then, there was nothing close to it, so it's no wonder they are so enthusiastic about them. Things are a bit different now - other languages have caught up.
ReplyDeleteMacros are great for conditionally ignoring a section of code. The one particular use I have in mind is for debug logging. Our massive code project is littered with function calls to do debugging, protected by `if trace_level` conditionals to prevent evaluation of the function call and its (sometimes obscenely large) argument payload. Two of the most expensive things one can do in Python are create objects and call functions, and logging generally does both.
ReplyDelete@Scott: I suppose I can see using macros to differentiate between debug and optimized code, although it looks like you have a passable workaround with simple Python conditionals. It seems that a macro facility would probably make your "massive code project" code look cleaner (i.e. without the conditionals).
ReplyDeleteBetween that example and collections.namedtuple, I'm just about convinced that macros could add something to Python. After reading the comments here, however, I am also convinced that Python needs them a lot less than a minimalist language like Lisp.
The point of macros is that you can extend the language without cooperation from the core developers. Consider the new Python "with" statement. If Python had macros, not only could this have been implemented outside the core (rather than waiting for a committee to debate it for months), it could easily be backported to older versions of Python.
ReplyDeletePython has such a rich array of features precisely because it must. Since it's impossible to extend the language in any significant sense, it must implement every useful feature it can. Some of us think of this as "bloat".
...and now I see MetaPython... nice work! Don't know if it's enough to keep me in Python-land for the long-term, but definitely cool.
ReplyDeleteI'm just starting Python, and I already found a good use case for Macros.
ReplyDeleteI want to run a test which is expected to raise an exception and write it like this:
test-exn("exception content", func(args));
Namely, I want this to expand to a try/catch statement without having to write the try/catch, and without having to write a class to encapsulate unit tests. These are for small scripts, but they're large enough that I want to have a small bit of regression testing. Perhaps I'm writing a miniature interpreter...
Lisp macros can (and do) do this.
@jeremy:
ReplyDeleteThe way this is typically done in python is to write an assertion function (this is how the unittest module does it):
assert_raises(ExceptionType, func, *args, **kwargs)
where the assert_raises function is something like:
def assert_raises(t, func, *args, **kwargs):
try:
func(*args, **kwargs)
assert False, '%s not raised' % t
except t:
return
except:
raise
But I can definitely see how macros could be useful, as well.
http://www.cs.brown.edu/~sk/Publications/Talks/SwineBeforePerl/
ReplyDeleteWatch the presentation and the slides about implementing a Finite Automata using Scheme Macros.
decent for macro'ed with a while
ReplyDeleteI want to replace for with a parallel_for that does something else with the block underneath the for. In order to do that with "with" I need to bracket the "for" with "withs"
ReplyDeletewith parallel_sync():
for i in range(0,4):
with parallel_split():
...closure that will have each
instance of the "for" run in
...parallel
This is the only way I can know,
1)each time I enter and leave an individual loop of the for block
2)when I leave the entire for block
I found this post because I wanted to implement functional conditions so that I could use them in a lambda statement, because
ReplyDeleteifelse(cond, t, f)
reads easier than
cond and t or f
People dont usually expect they will be used like that.
The problem is that defining them as functions cause both t and f to be evaluated before the condition. All the arguments are executed, and the return values passed into the conditional function, where the cond is then used in an if, and the appropriate return is returned.
Because of this, I realised that I wanted to use a macro to declare the function, using the and or trick.
I've kind of abandoned this project (it's still available at http://code.google.com/p/metapython/ if anyone's interested in looking at/picking it up). For something like ifelse, recent versions of python gave you the ternary expression:
ReplyDeletet if cond else f
which is marginally more readable (and more semantically correct) than the old
cond and t or f
Anyway, that's my $.02 worth.
Macros are for extending your compiler. I can implement a syntax transformation that can drastically change the semantics of a certain piece of code, without having to actually go in and hack the compiler to support the new feature in question.
ReplyDeleteFor example, I hacked together a simple tail-recursion macro today just for the hell of it.
https://gist.github.com/3967055
Obviously pointless and just for fun, but I know of few languages that could support something like this without a new version number.