SQLAlchemy: beware of backref

Wow, it’s been a while… Sorry, I’ll try to do better in the future.

I’ve been using SQLAlchemy for quite a while now, and I’m really enjoying it. According to Michael Bayer, the main developer, it’s modeled after Hibernate which is the best ORM I’ve ever used. There are a few things that SQLAlchemy does differently from Hibernate, one of which is “backrefs”, or automatic two-way relationship management. With Hibernate this must be done manually. Here’s a short example:

mapper(User, properties=dict(
    addresses=relation(Address, backref="user")

user = User.get_by(id=4)
address = Address.get_by(id=12)

assert len(user.addresses) == 0

address.user = user
assert len(user.addresses) == 1

The backref="user" causes Address objects to get a user property. And when a user is associated with an address the corresponding user.addresses collection is automatically updated.

However, there’s a subtle gotcha: setting address.user causes the user.addresses collection to be loaded. So if you’re associating a new child to a parent that has many children, then all of those children will be loaded when child.parent is set. Here’s an example:

parent = Parent.get(1)
# assume this parent has 500 children in the database
# since parent.children is a lazy collection they
# will not be loaded until parent.children is requested

child.parent = parent
# this implies parent.children.append(child)
# which causes all 500 of parent's children
# to be loaded from the database...ouch!

That’s annoying, and it causes an extra database hit which is expensive. Luckily there’s a fairly easy workaround. Instead of using the backref feature just define each side of the relationship separately. Here’s an updated version of the original example:

addr_mapper = mapper(Address)
mapper(User, properties=dict(
addr_mapper.add_property("user", relation(User))

user = User.get_by(id=4)
address = Address.get_by(id=12)

assert len(user.addresses) == 0

address.user = user
assert len(user.addresses) == 0

Notice that user.addresses is not updated when address.user is set. That’s because the two relationships are defined independently of eachother.

TurboGears – First Impression

The good

To be fair, I really only read the 20 Minute Wiki tutorial and Getting Started with TurboGears

TurboGears has a great set of components. This framework has promise. I’ve used SQLObject and FormEncode–both very nice. CherryPy is my favorite web sub-framework. I say sub-framework because IMHO it provides a good base for a web framework, but not much more. I’ve also looked at Kid and think it has a lot of potential.

I really like how easy it is to install. This should help dramatically with acceptance. I also look forward to keeping it up-to-date with easy_install -U.

Not only is it easy to install, but it’s very simple to get a base project up and running. I’ve heard about how Rails does this, but never experienced it in real life (that’s a different subject). Anyway, back to the quickstart feature. I can see this being very helpful, especially in creating a common structure for each project. Consistency enhances maintainability.

The documentation looks excellent. The writers have gone above and beyond the call of duty by not only documenting the new API’s introduced by TurboGears, but also by providing enough on each sub-component to get those unfamiliar with the sub-component(s) up to speed without leaving the TurboGears website. This is especially helpful in gaining an understanding of how each component fits into the overall framework.

Finally, it’s really simple to learn, and there’s no substitute for hitting the ground running!

The mediocre

Redirects are used heavily for control flow. I’m not very fond of this approach. I’m coming from Springframework where redirects were discouraged except when absolutely necessary. The reasoning behind this is that it’s better to simply render a different template than to invoke a roundtrip to the client and back just to get a different view. After all, the job of the controller is to choose the correct model and view for a given request. In fairness, this is only a development style, not a requirement of TG.

I’m not in love with the turbogears.flash mechanism of displaying a message on a redirected page–using a cookie. I’m not saying it’s bad, I’m just not sure it’s good to rely on something that some people have disabled to transmit messages within your site.

It doesn’t support Python 2.3 (yet), which means OS X users can’t use it without upgrading to Python 2.4 (come on Apple, get with the program).

The bad

UPDATE: The following no longer applies. A fix was committed within 12 hours of when this review was originally posted – greate work guys.

Automatic JSON output via /page/path?tg_format=json looks cool, but it could be a security vulnerability. Sometimes I send objects of which I don’t intend the user to see every detail to my page templates. With TurboGears, the user only has to add ?tg_format=json to the end of any TG controlled URL to get the objects returned by the controller method. The JSON output should be disabled by default. It could then be selectively enabled by some declarative procedure such as @turbogears.expose(json=True). I would expect most sites to expose only a subset of their URL’s as an AJAX API.

The ugly

raise cherrypy.HTTPRedirect(turbogears.url("/redirect/here"))

Exceptions are for exceptional situations and should not be used for application control flow

A redirect is NOT an exceptional situation. Therefore: thou shalt not raise an exception to redirect to a new page. A redirect is a tool to prevent the user from refreshing a submitted page (and thereby resubmitting the data, i.e. after a save operation) or to change the URL in the client browser window. It should be used sparingly, but it does NOT have “exceptional” status.

A better way to handle this would be to place a turbogears.url object in the dictionary returned by the controller like this:
return dict(tg_redirect=turbogears.url("/page"))

Questions and Answers

These are questions I thought of while reading the 20 Minute Tutorial. By the time I was half-way through the Getting Started page they were answered.


Out of the box, TurboGears provides everything needed to create a complex web application. One simple easy_install command and you’ve got it all. With a little more effort I think TurboGears could be one of the best web frameworks available for Python.

Safety Language?

“The popular safety languages are C++, Java, C#, VB and Delphi.” – Kevin Barnes

I disagree with one part of that: VB is not a safety language. Rather, I would say “Visual Basic is a danger language.” It has the verbosity and cumbersomeness of a safety language, but does not give the one important benefit of a safety language: safety. It’s just plain dangerous. Hello Variant–can we say “not type-safe?” By definition, a safety language should enforce type safety, which VB does not do.

To be fair, I have not used VB.Net, but who would when they can use C# (a true safety language)? Personally, I prefer freedom.

AOPython 1.0.2

AOPython is an AOP (Aspect Oriented Programming) module for Python. AOPython provides a base ‘Aspect’ that can ‘advise’ or wrap function and/or method calls. It also contains a weave function that can weave (apply advice to a subset of- or all functions and methods in) a module, class, or instance. Advice may modify or replace parameters and/or the return value of the advised function or method each time it is called. It can do many other things such as handle exceptions raised by an advised function, collect information on the method call (logging), or obtain and release resources before and after a method is invoked.

Download AOPython 1.0.2.

Example 1: How to wrap a function

from aopython import Aspect
def negate(x):
    return -x
aspect = Aspect()
negate = aspect.wrap(negate)
negate(2) # advised function call

Example 2: How to weave a class

from aopython import Aspect
class MyObj(object):
    def double(self, x):
        return x * 2
    def tripple(self, x):
        return x * 3
aspect = Aspect()
myobj = MyObj()
myobj.double(5) # advised method call
MyObj.tripple(myobj, 5) # advised method call

In this release the weave functionality has been enhanced to allow more flexible weaving of callables in modules, classes, or instances. An unweave function has also been added to allow advice to be removed en-mass from an object that was previously weaved. Test coverage was also greatly improved, especially for the weave/unweave functions.

AOPython 1.0.1

Version 1.0.1 of AOPython is now ready for consumption. If you’ve been using 1.0 I recommend that you upgrade to this version as the API has changed slightly in a non-backwards-compatible way: the order of the first two arguments of the weave function has been reversed (that’s all).

Otherwise, the documentation is improved slightly and there are a few other minor changes. See the changelog for more detail.


AOPython 1.0

I’ve written a small module called aopython: Aspect Oriented Python. I just wanted to make it public (mostly so I can download it at work). I’ll write more about it later, but for now here it is. Please feel free to make suggestions if you have any.

Python and dynamic typing

While reading Typing: Strong vs. Weak, Static vs. Dynamic my mind was jogged by this statement:

Pythonic programming style is to use inheritance primarily for implementation; Python’s name-based polymorphism means that you rarely need to inherit for interface

I was reminded of the Java Toolbox article on Why extends is evil. What implications does the “The fragile base-class problem” have with regard to Pythonic style?

Q. Is Python evil because extension is the only option (there is no such thing as an interface in Python)?

A. No. Python objects can do what that article is talking about: wrap other objects to encapsulate behavior without exposing methods with incorrect semantic meaning.

One thought is that Python developers have no need to explicitly define interfaces, they just write code that implements them. I like less code, but I’m still grappling with the idea that, in Python, there is no easy way to see where an interface change will break other code without writing comprehensive unit tests. With Eclipse a change to the interface and a recompile (automatic) will reveal most if not all breakage points within the application (assuming minimal use of reflection and introspection). One very useful aspect of this behavior in Java is the ability to get an idea of the impact of a given interface change–and Eclipse makes it really easy to see that impact as well as to fix the damage once the change is made. Having said all that, interfaces are no substitute for good unit tests. Ahh, but they sure do provide a nice false sense of security…hey, it compiles. With Python the statement “hey, it starts up” only means that the code needed to get your application started is working. Could a tool be written to test Python code for the type of errors that are exposed by Eclipse when a Java interface changes before the change is resolved in the implementation?

Maybe an “interface testing module” could be written to detect if an object implements a given “interface”? …but now I’m back to writing my interface in code … I guess any comprehensive test suite would most likely have that information anyway.

For reference: encapsulation

Interesting, but not really related:
Why getter and setter methods are evil and More on getters and setters. Future post: how does the getter/setter problem in these articles relate to Pythonic style?