There’s been a lot of swirl around Ian Ayres, an econometrician and law professor at Yale, and his new book, Super Crunchers, and I’m generally in favor of it. Ayres advocates using data, not intuition, to make decisions. “duh” you say? Me too, but apparently there are lots of people who prefer their own intneral compass to the votes of customers. It’s no coincidence that the issue of Newsweek that reviewed Ayres’ book has Alan Greenspan on the cover.
In this wired world, we’re swimming in data – log files, CRM, registration forms, hits, clicks, sessions, the works – but not enough decision makers have the desire or the tools to turn those data (yes, they’re plural) into actionalbe information. They see number crunching as easy as miss out on the most important part, the experimental design. A recent article in Newsweek about Super Crunchers contains an instructive example.
Ayres chose the title of his book by running two Google ads that appeared in random order when someone searched for phrases like “data mining.” The decision was made by the plurality who clicked on the ad for “Super Crunchers” rather than the competing title, “The End of Intuition.”
People looking for an easy way out will see this as justification to test everything with anonymous masses, let majority rule and never speak with another individual human again. But they will find themselves in a poultry-ovum priority quandry – how do you choose which candidate names to test without first testing those names? And it gets worse – how do you know if Google adwords is really polling the right people, in this case potential buyers of your book? Is the result with 500 hits really better than the one with 450 hits?
You can take the intuition out of evaluating an experiment – and I think you should – but you can’t take the essential creativity out of designing a good experiment in the first place.
Remember, friends don’t let friends crunch numbers blind.