FriendFeed has an in-house framework inspired by web.py and Appengine's webapp. It was written by Bret Taylor, who was the PM of appengine when it started, but is now working for FriendFeed.
He mentioned that he might open source it, but hasn't done so yet. I encourage people to put some pressure on him to do this =)!
Tipjoy is my first web project, actually. I started knowing python. I tried frameworks, and quit when something failed. Django was the first to go from start to live test site touch using a DB without a hiccup. I think that means something for both the quality of the documentation and the framework.
TurboGears, web.py, and Pylons failed the test.
The Python community would benefit from more people switching to Django. It does have the approval of the BDFL, after all.
FWIW, that isn't a fault of the framework per se; I have had no problems with Pylons doing something it shouldn't have done. You have to understand what the code is doing, and each framework has a different learning curve, but in my experience Pylons is as usable, expressive, and understandable as Django.
I'm certainly just a single data point. The difference between Pylons and Django could have been documentation alone, and could have been unique to me.
I swear by werkzeug.pocoo.org -- not a framework, just a neat abstraction over WSGI with lots of web-related utilities and HTTP internals support inside that keep coming in oh-so-handy.
I evaluated Django, TurboGears and Pylons for a new-from-scratch application of medium complexity (10K lines Python, 4 different databases). Pylons seemed to use a number of other tools well, had an excellent book and helpful community, so I went with Pylons. Turned out fine for my app.
For anyone who considered Django and discarded it (especially recently since it's been 1.0), could you maybe explain the decision behind that? I'm curious as a relatively recent Django convert.
It seems that its support for SQLAlchemy isn't really first class. I'm sure its builtin ORM is workable for simple web apps, but if you are doing serious database work in python, there is no substitute for SQLAlchemy.
Of course, you can use whatever other libraries you want in your own code, but at the point where you are not using Django's ORM and you have left its lame template system for something like mako (or maybe genshi, especially if you are generating XML), then what is Django really giving you?
I want to preface this by saying that Django isn't bad. It's really good. However, there are some maddening things.
For example:
In Rails, you can say
Article.find(:all, :include => :comments)
and that will get you the articles with their comments. In Django, you can say
for a in Article.objects.all():
a.comments #hits the database again! argh!
Before you say, "Django has select_related() which does the same thing as Rails!" select_related() doesn't work in any useful way. select_related() only works on the side that defines the foreign key - so you can get the single article that a comment belongs to, but you can't get the comments on an article without running another query. Yeah, that simple, basic, every ORM should support it case isn't handled. And it's maddening.
Forms handling and validation is another sore spot. Some validation happens in the model, some in the form, ModelForms does somethings, but not others, editing related objects is a mess. In some ways it's shocking that it's such a mess considering how awesome the admin interface is. However, I always find myself manually dealing with forms to a point that's just annoying.
The lack of a true "if" in the templates. It's just limiting in a way that I don't find useful and I find that it makes my code less clear by nesting things in weird ways to get the desired effect.
Stagnation. Django is awesome, but sometimes it takes forever for small things to go anywhere because there's a genuine hesitance in the community to "dictators". By dictators, I mean personalities like DHH who say, "well, this way works and is elegant and we're going to go with it." On the one hand, it's what makes Django really attractive as a community. Rails attracts the know-it-all personalities. Maybe DHH and Zed do know it all, but most Rails programmers definitely don't. People in the Django community are really helpful, respectful of other's knowledge, and love learning from each other. However, it also means that even tiny features must be rehashed over and over, filed in triplicate, lost, found, stamped, checked by a committee of 8, rewritten from scratch, named new-X or X-refactor, break API compatibility, and finally get into SVN. Ok, most of that was a joke, but there are lots of little areas where it just seems like Rails is willing to move on something based on the general mood where Django wants a more formal assessment and wait period before deciding whether something is ok to be in or not. And for all people complain about DHH's one-way-to-do-it mentality and Django's more open structure, Rails isn't so closed and Django not so open.
Django's big selling points (for me) are that there's a ton of python out there that's beautiful, reliable code and the admin interface that allows you to deal with content neatly and easily. Clearly there are other selling points like MVC and ORM and whatnot, but there are many frameworks that have that. Django is a lot easier to understand (at least for me) in its implementation. Django's documentation is top notch which is amazing compared to other projects, but Rails is catching up with their guides and screencasts. Django also tends to document things like their file backend hooks better which makes creating plugin-type things easier, but again Rails (with the help of the Merb team) is going to be a ton better in this area.
You won't go wrong with Django. The only fatal flaw is really the first one and it's just unacceptable at this point in time. But it's a great set of code with a wonderful community. I don't think there's another framework I'd prefer. Maybe Rails, but nothing else.
Your first issue is incorrect. You can easily accomplish the query with one sql hit. The extra() command would work if your foreignkey was defined in the comments model.
Nope. extra() doesn't allow one to do that. extra() can be used to populate an attribute. So, say I wanted to do a comment count I could say
Article.objects.all().extra(select={'comment_count': 'SELECT count(*) FROM blog_comment WHERE blog_comment.article_id = blog_article.id'})
But I can't populate a list of comments objects onto it.
I've talked to Malcom about it (who wrote the QuerySet Refactor branch - or at least a substantial portion) and it's a known deficiency. Basically, the issue is two-fold. First, no one has stepped up to write the code that would make select_related() work in that fashion. Second, people want the implementation to disallow certain bad situations.
The first part is self-explanatory: select_related() should be enhanced to support that, but someone needs to write the code. The second part isn't as much, but it's more interesting.
Let's say you execute this query:
SELECT * FROM articles LEFT OUTER JOIN comments ON comments.article_id = articles.id
How many rows will you get from that? You'll get somewhere in the vicinity of the number of comments (adding a row for any article without comments). Suffice it to say, the results set grows linearly in proportion to the number of articles and comments.
Now let's say we have this:
SELECT * FROM articles LEFT OUTER JOIN comments ON comments.article_id = articles.id LEFT OUTER JOIN votes ON votes.article_id = articles.id
So, we have our article with comments, but also votes now. So, let's say we want to get just one article that has 100 comments and 200 votes. How many lines will that return? In the original SQL query, we would have seen 100 lines (one for each comment) which would have been manageable. Here, we get 100 * 200 lines back. That's 20,000 rows to parse to build a single Article object!
So, the Django folk aren't into letting you just hang yourself out to dry like that. There have been proposals to limit it in ways that wouldn't let you do that, but it's really just something genuinely missing and there isn't a way to do it other than running looped queries or doing something ugly.
extra() doesn't do the same thing as select_related(). I'm not saying that doing ORM over multi-valued relationships is easy or that you can't execute queries that are really bad doing them, but it still means that there isn't support for a pretty basic function.
Wow, that's an excellent response. You're right. I can't think of any means of easily accomplishing that - if at all; which I find odd, because I would think that would occur at least a few times.
Grok's main value, to me, is to interact with larger Zope applications without having to deal with all of the mental overhead that Zope entails.
You can also reap the great benefits of the ZODB very easily with Grok alone. Grok is very easy to learn, and was designed by people that really know what they are doing with regard to Python web app development.
Thanks, I've heard Zope is huge and that must be the mental overhead you were referring to. I take this to mean that it was at once a large project, but jusdging be the 124 to django and 4 for Zope + Grok I take it that it has fallen out of favor. Why? Does it make anything too difficult? Is it too closely married to ZODB, ie. are there problems with ZODB?
Zope's main problem is one of branding. Zope3 is really an entirely different beast than Zope2, but the sins of Zope2 (chiefly a monolithic and deeply intertwingled codebase that meant you mostly had to swallow the kool-aid to use any of it) have continued to plague Zope3's acceptance, even though Zope3 doesn't have the same problems.
Still, Zope3 requires a bit more mental overhead due to the infrastructure it contains meant for handling extremely complex application domains and use cases with relative ease. For example, none of the other frameworks has an authentication and authorization implementation that is anywhere near Zope's in terms of pluggability and flexibility. Very useful if you need to build an intranet app that gathers user login credentials from several systems, and their permissions from some other system, but almost entirely useless if you're building a Web 2.0 app where content is strongly owned by an individual user who only needs to grant viewing permissions to other users.
At least, until you want to add group permissions, role-based security rules, or need to alter permissions based on workflow state, and be reasonably sure that your solution is actually secure.
The Zope universe is full of reusable optional libraries that solve problems that are this complex, and the framework itself has to have some additional complexity in order for it all to plug together in a reasonable way. This extra level of complexity is represented in the Zope Component Architecture (primarily zope.interface and zope.component). To handle the 'plugging together' part as configuration rather than code, we also have an XML-based configuration language called ZCML.
To people who think that they don't need that complexity because their problems are simple, Zope seems like overkill. But then they end up re-inventing wheels such as access-control-lists, and not necessarily doing it well at all.
CherryPy is a hugely underappreciated project in the Python community.
Its HTTP implementation is top notch, its tool system is a great way to write custom modules, it has excellent WSGI support, and it's just about the fastest HTTP server in Python that takes a thread-based concurrency approach.
I was the first engineer hired on a team that built an online survey-research company. We started using twisted/nevow, but eventually moved to an in-house web framework on written on CherryPy. Lots of tools have come and gone (mostly, we've rewritten components we found lacking) but CherryPy has never let us down. We even ended up hiring a large portion of the CherryPy team.
Our experiences with it were positive enough that I'm building a startup right now on a similar CherryPy-based stack (at least partially--there's also a bit of Erlang involved for the high-concurrency stuff).
If you're considering stitching together your own Python Web framework--as many shops eventually do--I highly recommend CherryPy be used as the foundation.
Bad form replying to myself, but one more thing worth clarifying:
All the above being said, it's only fair to acknowledge that this isn't quite apples-to-apples when you look at CherryPy vs. something like Django. CherryPy isn't really a full-stack web framework, it's an HTTP server attached to a configuration and tool system that facilitates the creation of custom web frameworks.
As odd as it may seem we often prototype something we need to test using web.py even though our primary development language is php (using Zend). I recently did this to test a REST API. This has also been part of our process of 'flirting' with using a python web framework for some of our applications (looking at pylons).
We use Nagare (http://www.nagare.org) for our most important customers projects, A framework with a very different approach to web development than the classic MVC / url routing / session object / template frameworks.
The caching framework is really cool. I did a test a couple of weeks ago, and I got a 10x performance boost, simply by sprinkling 3-4 lines of caching code to my site. That was justing the local memory caching: http://docs.djangoproject.com/en/1.0/topics/cache/#local-mem...
I'd imagine that using a caching back end like memcache would probably give another huge performance boost. It's really pretty straight-forward and usable out of the box.
Ellington does not run the entire Washington Post site. A number of specific applications on washingtonpost.com use Django (e.g., projects.washingtonpost.com/congress), but the main site (articles, columns, etc.) does not use Django.
Signed, the guy who introduced Django to washingtonpost.com
I'm not disagreeing that scaling isn't only caching, but it's often a component just like anything else. Caching often helps scaling, as do multiple databases, working out bottlenecks in code...whatever else.
Scaling = Increasing capacity indefinitely by adding more nodes.
Caching is not scaling because by definition, it depends on a backend to provide it data.
If the backend is a database then sure, you can increase capacity by ~10x (assuming a ~90% cache hit rate), but after that, throwing more caching nodes at the problem won't do a thing if your database itself isn't scalable.
All a cache does is increase speed at decreased costs, but it is not in itself a scaling solution.
For example, did you modify the source to support multiple databases for read/write? Or is that not an issue and it's OK that it's coming later in the framework's roadmap?
As I see it, if we get enough traffic to be worried about scaling issues we'll deal with it then. Django isn't terribly designed and going in and hacking multi-db and sharding in should take less than a week from a competent developer.
Discussing this before starting a startup sort of feels like trying to figure out how to do salary for hundreds of employees before getting your first one.
I agree it's worth discussing (especially if you're unsure about future malleability of the web framework to your needs), and I apologize if I was excessively curt.
The framework may not offer it as part of the framework, but there is nothing stopping you from doing it yourself. It's python, it's a database, what's the difficulty here? Django is not the type of framework where you run deep within it and you cannot break out - nothing stops you from doing anything you want to do.
What I mean is that you do not need to modify Django. You can just write a routine that duplicates whatever you are writing to the database manually. The Django process is easy to break out of from within your app.
Django has had low level, undocumented hooks for talking to multiple databases for ages (Malcolm talked about these at DjangoCon, the same conference that hosted Cal's excellent keynote). A very recent checkin makes those hooks easier to use: http://code.djangoproject.com/changeset/10026
Why is that surprising? General purpose frameworks are always suboptimal, and overly general. There's extra cruft you'd never need, parts you might need optimized aren't, etc etc.
Writing your own is often less work than trying to tailor something existing to your own needs.
it's very easy to write your own framework in python, that does excactly what you need it to do and nothing more.
therefore is:
scaleable the way you need it for your application.
not in your way of doing things.
easier to debug.
it's a bit of development overhead in the beginning, but pays of in the long run.
If I'm just coding up a small app I don't want the overhead of a framework. I might want a bit of routing, possibly some automated mvc and that's about it. Everything else in a framework I usually care little about and just slows stuff down for no benefit to me.
Coding up a "framework" that does only the very little I actually need is fast and very little development overhead considering the number of times I've done it and the fact that I've got code I can re-use.
This isn't business advice however, it's just my own personal project preference. If you're running a business you want to be able to replace developers if they quit so it's often a better idea to go with something more mainstream like a well known framework so that ramp-up time is minimized.
c) No single developer 'owns' any part of the code
That should ensure that your developer team's "bus number" is reasonably high.
Finally:
d) You treat them at least well enough that you're not in danger of the entire team quitting at once.
That said, it can still be useful for many reasons if they base their work on F/OSS code (languages, libraries, tools, frameworks, etc.) that has a reasonably-sized community around it. But if the developers you hire are good that is very likely to happen naturally.
There are always circumstances where writing your own may make sense in the short-term, but unless it has something to do with your business' core-competencies, or otherwise leads to a sustainable business advantage, you're usually well advised not to bother.
Although it's not on the list, WSGI is the framework. The most simple and flexible one (it must be, as all the other frameworks on the list are based on it).
Frameworks are great up to a point. If you burger flip web sites all the time, then they make your life easy.
If you only deal with one site, you will never get the flexibility of pure WSGI with some prefabricated framework.
Heresy! j/k. I'm actually curious why you switched from python to php. Was it your choice or forced upon you? If your choice, can you explain? I usually meet people going the other direction.
Well, I was running a small business at the time, supporting other small businesses and small sites. Virtually all of the ones I hadn't written were using PHP, so as I fixed problems and added features I found that I was building a library of PHP even though I built all "my software" in Python, Common Lisp, and a few other one offs. As I contemplated getting out of the business, it dawned on me that it was going to be terribly difficult and expensive for all these small businesses to find someone to support and extend the apps written in various languages I'd used, and there wasn't anything really novel or language-dependent in the CRUD apps I was writing (mostly because my clients were almost all mainly focused on the look of the site and graphics, which was all handled outside the main code via templating, and which I didn't do myself). So I began to write new things in PHP with an eye to divesting myself of the client base and getting a "real job", as my friends and parents would say. This led, in a very straightforward manner, to my getting PHP positions when I folded the business and moved to the DC area, last year. This turns out to have been fortuitous, since PHP and Java are the main languages represented in web development, at least around here.
Perhaps that was a longer answer than you wanted. :)
He mentioned that he might open source it, but hasn't done so yet. I encourage people to put some pressure on him to do this =)!
http://bret.appspot.com/entry/experimenting-google-app-engin...
EDIT: I'm not sure who wrote the framework, so I was wrong to say that Bret wrote it. In any case, it was written by FriendFeed developers.