This one was just so good I had to put it here - if you're already reading the pgsql-general list, just ignore this post. On a thread about stupid benchmarks (comparing databases with default settings without changing anything, comparing MyISAM with a transactional engine, etc), Tom Lane gave us this gem:
I think it's a fine analogy, precisely because they're both good in their respective problem domains. Try to carry 500 people from Los Angeles to Tokyo in an F-15. No? Try to win a dogfight in a 747. No? But they both fly, so it must be useful to compare them... especially on the basis of the most simplistic test case you can think of. For extra points, use only one test case. Perhaps this paper can be described as "comparing an F-15 to a 747 on the basis of required runway length".
That's a good one, Tom! And can we please have proper benchmarks instead of this kind. (I'm intentionaly not linking to the actual benchmark in question)
New comments can no longer be posted on this entry.