Sunday, March 11, 2012

Using decorators and closures in python

Decorators

A decorator wraps a class or method to extend it. In python the syntax used to wrap the original function is @decoratorname. For example let's say the original function A needs to be decorated with B:
def B(a):
  def C(*args):
    return 2 * a(*args);
  return C

@B
def A(x, y):
  return x + y;

print A(2, 3)
This makes A return 2 * (x + y) instead of x + y. So the result printed will be 10. What actually happens is that the decorator will return a new method C(x, y) which returns 2 * A(x, y) and will assign it to A. So the @ sign is actually syntactic sugar for:
def A(x, y):
  return x + y;

A = B(A)

print A(2, 3)
We could also make the 2 inside the decorator a parameter of B like this:
def B(z):
  def C(a):
    def D(*args):
      return z * a(*args);
    return D
  return C

@B(2)
def A(x, y):
  return x + y;

print A(2, 3)
What happens here is that B will return a new decorator C with a z parameter fixed on 2, then C will return a decorated function of A called D wich will replace A.

Closures

A closure is a function together with the non-local variables it references. C and D are closures as they keep references to a and z. For example A here returns a closure B in which the term z is fixed at the moment the closure is created, so for C, z is always 2:
def A(z):
  def B(x, y):
    return z * (x + y);
  return B

C = A(2)

print C(2, 3)
The variable cannot be reassigned later, but it can reference mutable object. The following code will print "cat", since the reference d is not changed inside the closure:
def A(d):
  def B():
    return d.get("a", None)
  return B

d = { "a":"cat", "b":"dog" }

C = A(d)

d = { "a":"tiger", "b":"dog" }

print C()
If however we change the dict d points to, it will be reflected in the output of C, so the following code prints "tiger":
def A(d):
  def B():
    return d.get("a", None)
  return B

d = { "a":"cat", "b":"dog" }

C = A(d)

d["a"] = "tiger"

print C()
A usage example

In makehuman we use both decorators and closures to obtain an easy delegate mechanism for events. For example to handle the event of clicking a button you have two ways to do this. One way is subclassing and overriding onClicked:
class MyButton(gui3d.Button):

  def onClicked(self, event):
    print "event %s handled for %s" % (event, self)
This is useful when needing many buttons doing a similar task. Like different buttons to choose different colors. You would pass the needed color to the constructor. In many cases however you don't want to spend time subclassing objects. In those cases we use the following decorator:
@self.myButton.event
def onClicked(event):
  print "event %s handled for %s" % (event, self.myButton)
This decorator allows us to create a button and assign a new method to self.myButton.onClicked, without needing to create a new class. The (very simple) decorator implementation itself looks like this:
class EventHandler(object):

  def event(self, eventMethod):
    setattr(self, eventMethod.__name__, eventMethod)
What it does is looking up the name of the method which is passed (onClicked in this case), and setting this attribute in the object instance. This replaces the existing method if there was one. Due to the handler passed being a closure, the self parameter or any other non-local variable is accessible for the handler. A possible improvement would be to use an extra closure like above to have the button in question passed as a parameter to the handler. The extra closure would avoid having to write self.myButton and making mistakes when copy-pasting handlers:
class EventHandler(object):

  def event(self, eventMethod):
    def closure(event):
      eventMethod(self, event)
    setattr(self, eventMethod.__name__, closure)
Using this we can write the handler code like:
@self.myButton.event
def onClicked(button, event):
  print "event %s handled for %s" % (event, button)
The reason self isn't passed originally is because the handler is not a class method, but simply a function because we assign it to an instance, not the class object of the instance. This is however desired since we want each button instance to act differently.
Another improvement, not really in the scope of this article, would be to allow multiple handlers for one event. This will be added to makehuman in the future, since it uses this for some events. The idea is to assign a class to the entry with the method's name which keeps the list of handlers and calls these in it's __call__ method:
class MultiHandler(object):
  def __init__(self, handler):
    self.handlers = [handler]

  def __call__(self, event):
    for handler in self.handlers:
      handler(event)

  def addHandler(self, handler):
    self.handlers.append(handler)

class EventHandler(object):

  def event(self, eventMethod):
    def closure(event):
      eventMethod(self, event)
    if hasattr(self, eventMethod.__name__):
      obj = getattr(self, eventMethod.__name__)
      if not isinstance(obj, MultiHandler):
       setattr(self, eventMethod.__name__, MultiHandler(obj))
      getattr(self, eventMethod.__name__).addHandler(closure)
    else:
      setattr(self, eventMethod.__name__, closure)
When assigning the first handler, we just store the handler. Assigning a second one will create a MultiHandler instance which keeps a list of handlers, the exisiting handler is passed to the constructor. Afterwards each additional handler is added to the MultiHandler. Since the MultiHandler supports the __call__ protocol, nothing special needs to be done to support this additionally to the simple method handler, and there's no extra overhead.


No comments:

Post a Comment