I make it a habit to stay away from databases these days, but I tripped over this (slightly) old blog entry from Dave DeWitt and Mike Stonebraker on Map-Reduce. I like Dave and Mike’s thinking, but then I guess I’m a database gray-beard as well. I do have one quibble with their analogy. If Codasyl database programming was assembly language (and I have plenty of experience with both) then MapReduce is closer to microcode.
It’s not that I don’t think MapReduce is useful, it’s that I see it as a failure of the database community to deliver what we keep promising. We can talk about Teradata and a few other "grid" offerings, but for the most part we’ve done a terrible job of productizing grid/cluster/parallel database technologies. Yes, we’ve certainly known how to do it since the 80s. But today’s mainstream products either don’t offer it or offer it in very limited, and indeed very high priced, forms. With most technology we would have had the specialized high-priced offerings in the 80s and by now it would have ridden the learning curve to the point where the technology was super-cheap and ubiquitous. I know why we haven’t gone anywhere on this, but that’s the topic of another post.