I keep thinking I’ve posted my last article about the SNP’s meretricious ‘National Survey’ (this links to all my previous posts on the subject). Then along comes another scrap to keep the pot bubbling.
Since I last wrote, the closing date has passed to contribute to the survey/give your contact details to a marketing exercise. Three things have occurred almost simultaneously.
First, a YouGov poll based on fieldwork undertaken 25-29 November asked this question:
Over the last three months the SNP have been carrying out a “National Conversation” survey on independence. Have you been approached to take part in the “National Conversation” survey? [on p.6 of the YouGov report]
The results were:
- Yes – 13%
- No – 82%
- Not sure – 6%.
Let’s try to work out what this might mean in the population as a whole.
Start by assuming that everyone who said yes or no was telling the truth.
Assume that the 6% who said they were not sure had not been approached: if they had been I suspect they would remember.
Errors of estimate in surveys of this sort are typically ±3 percentage points. So that 13% who said they’d been approached to take part could be as as low as 10% in the population as a whole, or as high as 16%.
Let’s also assume that the ‘population’ means, for this purpose, the electorate for Holyrood elections/the 2014 referendum, that is everyone aged 16 or over. The most recent available official estimate for this group of people is 4,460,000 (mid-year estimate of population, 2015 [p.15]) .
Therefore, the likely number of people approached to take part in the survey would be between 446,000 at least (10%) and 713,600 at most (16%).
Note that YouGov asked whether people were approached to take part, not whether they did take part. Some will doubtless have said ‘No thanks.’ Others, on circumstantial evidence appearing on social media, will have taken part more than once using different names.
Nevertheless, as I say, the maximum number of real people approached to contribute was most likely no more than 713,600. Here is the SNP’s claim of how many responses they received:
Based on the YouGov survey, this is 2.8 times the maximum number of people likely to have even been approached to take part. To put it another way, the SNP claim 45% of the entire adult population of Scotland responded to the survey.
This is remarkable and frankly I don’t know what to make of it. The possibilities seem to me to be four:
- the YouGov estimate is wrong, but wrong by a margin of error hugely larger than any other properly conducted poll I’m aware of on any subject in modern times
- those people who filled in more than one form did so on average a humongous number of times to bridge the gap between 446,000/713,600 and 2,000,000
- the SNP mean something radically different from a ‘response’ than I would understand, maybe adding in people who attended meetings or talked to them in some other way or took part in other recent data collection exercises they’ve undertaken, or … what?
- someone’s telling porkies.
Second, when they first started this survey the SNP got into trouble with the information commissioner’s office for misrepresenting the nature of the survey, intentionally or unintentionally. I’d argue that there is now another sort of misrepresentation. Detached from the tweet, as it will be many times, the graphic above still contains no reference to the political party that carried it out, still includes the misleading ‘National Survey’ logo, and styles Nicola Sturgeon as ‘First Minister,’ her government role, not as ‘SNP Leader,’ which is what she is for this purpose.
Third, and I’d welcome a definitive statement on this from the SNP, there have been widespread claims in the mainstream media and on social media that the SNP won’t even be publishing the results of the survey. I put this claim in italics because it strikes me as frankly outrageous if true. What came in with a bang on the tailcoats of the non-existent summer of wooing and became characterised as ‘The biggest listening exercise in the SNP’s history’ would go out with a whimper amid suspicion that whatever it was, it wasn’t what it claimed to be.
I don’t know how many people took any notice of my original scepticism about this data-gathering exercise but I confess to feeling vindicated in my original advice to ‘avoid it like the plague.’ If the SNP are right, 2,000,000 people disagreed with me. But I wouldn’t bet on it.