The Devil Is in the Details

When I saw a link to a survey by ScriptLogic (PDF) showing that Corporate America™ was not planning to speedily adopt Windows 7, it took only a few seconds to think up a witty post about it. But then I actually read the survey:

Survey Details

• Received 1100 Respondents

Um, that’s it? Firstly, the use of the plural in this section heading is entirely unjustified for there is, in fact, but one detail. Secondly, the word “respondents” makes me suspect a convenience sample rather than a truly random sample. ScriptLogic probably just mailed a questionnaire out and compiled the results from those questionnaires that happened to come back. And therein lies the problem: the decision to fill out or not fill out the questionnaire has acted as a selector of responses and therefore has de-randomized the sample. It is probable, after all, that the questionnaire recipients most likely to bother responding were those who are fed up with Windows and unenthusiastic about its upgrade. Concurrently, those who are content with Windows and who plan to upgrade in due course were probably less likely to respond.

Of course I don’t know if that is how the survey was conducted and that, ultimately, is the salient point: How can I trust a survey whose methodologies are so opaque? There may well be widespread ennui about Windows 7, but this survey is insufficient to prove that.