‘New poll shows 50% of Scots want Indy!’

This was the headline picked up by most of the media from a recent opinion poll on the issue of independence/separation, commissioned by STV from Ipsos MORI. The truth behind the headline is different and this post attempts an analysis of that number absent in most comment so far, and entirely missing from the triumphalist tone of most nationalists online.

I don’t normally devote time and effort to a detailed analysis of any single poll as the trend over time across all polls is what’s important. However, because of the praise lavished on this one by enthusiastic nationalists, not to mention the gloom it has induced in a small number of pro-GB Scots, I am happy to make an exception.

The Ipsos MORI poll comprises an unusually complex database. To check the full facts of what I’ve extracted here you may wish to go the company’s website, where (towards the bottom of the page) you will find three separate documents referenced – what they call the survey ‘topline’ giving a summary of the results; a full set of output in 29 separate ‘computer tables’; and a more user-friendly set of colourful charts. I draw on all three sources for these comments.

First, a statistical health warning that Ipsos MORI don’t spell out, except obscurely in the tiniest of footnotes to their computer tables where they flag up some results as ‘very small base (under 30) ineligible for sig testing.’ What they’re referring to here is that all properly-conducted sample surveys, and I’m sure this is one, have a margin of error associated with their estimates. Typically with a survey of about 1,000 respondents (this one has 1,029) that margin of error is about three percentage points. So straight away, the true value of an estimate of 50% could lie anywhere between 47% and 53%. That’s no comfort to either camp in what seems like our interminable debate on separation, but it is a fact.

Ipsos MORI’s pie chart at the head of this post summarises their finding. To see the detailed source of the claim that ‘50% of Scots want indy’ you have to go to Table 3 in the ‘computer tables’ (click to enlarge or check it on the Ipsos MORI website):

(The meaning of the codes in the two rows in bold, ‘bfjm’ etc is not clear)

You’ll see the 50:50 breakdown in the left-hand ‘weighted’ column. The ‘unweighted’ figures on the right show a slightly different result.   For want of a better phrase weighting is a way of adjusting a particular sample to the known characteristics of an overall population. Properly done, it is quite legitimate, and it would have been helpful if the company had provided information to show how they weighted this particular sample. Without that adjustment there is a small but clear majority in favour of ‘No.’

When you look at the 50% figure you also need to be aware that it is not a figure based on the 1,029 respondents:

  1. it excludes people who said they didn’t know how they’d vote, or declined to say how they’d vote
  2. it only includes those who said in answer to another question that they were highly likely to vote in another referendum (9 or 10 out of scale of 1-10).

The bottom line is that both weighted and unweighted estimates are based on 858 people, not 1,029 (i.e. on 83% of the total sample), and any error of estimate will be slightly higher than the ±3 percentage points I mention above.

It’s also worth noting that the full question people answered was:

If a referendum were held tomorrow about Scotland’s constitutional future, how would you vote in response to the following question: Should Scotland be an independent country?

The ‘if … tomorrow’ formulation is a common one in political opinion polls but of course it’s not realistic. This poll didn’t ask another critical question – do people actually want another referendum now? An analysis of recent polls by Professor John Curtice that did ask the question suggested that only just over a third of Scots want another referendum ‘any time soon.’

The Ipsos MORI poll does have one interesting feature many similar polls lack. It not only asked the Yes/No question, it also asked people:

on a scale of 1 to 10 where 1 means ‘I completely support Scotland becoming independent’ and 10 means ‘I completely support Scotland staying part of the UK’ , what number would you consider yourself to be? (Table 9)

So a more subtle analysis of people’s strength of feelings either way is also possible. Here is Ipsos MORI’s summary table of that information, annotated by me (the brackets and summary percentages at the bottom):

Two conclusions can be drawn from this:

  1. the core support for staying part of the UK is significantly higher than the core support for independence – 38% vs 28%
  2. if you divide the whole sample into two, those  tending to want independence (1-5), and those tending to want to remain part of the UK (6-10) you find a split remarkably similar to the 45:55 of 2014.

There are surely Scots on both sides of the Yes/No divide as well as those with no settled opinion who will say ‘Is that all the turmoil of the last 2½ years has produced, virtually no change from the last referendum?’ To which I would, of course, add the statutory reminder ‘once in a generation’ (©Alex Salmond/Nicola Sturgeon).

All this, and much more (see for example Nine reasons why my money’s on no more Scottish referendums …), makes me conclude that the first minister would, excuse me, be a damned fool to commit herself to another separation referendum any time in the near future.

This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

4 Responses to ‘New poll shows 50% of Scots want Indy!’

  1. ProUK AntiTheism says:

    Being damned fools has never stopped the SNP before unfortunately.


  2. Stuart says:

    She may be a ‘damned fool’, but events dear boy, events……

    As she has threatened it so often, that she may be compelled to call indyref2.


  3. Robert says:

    Roger…In the eighties, I spent some years carrying out postgraduate research into error in quota sampling, the method used by MORI and other agencies to complete these surveys. All the variability assumptions/calculations are based on the purer ‘Simple Random Sampling SRS’ In reality the variability of their sample could easily be + or – 5% and is unlikely to be at the SRS level quoted. These polls are too variable to be trusted and should be banned as the odd bad one just feeds the Nationalist bandwagon.

    Liked by 1 person

  4. Comparing Polls taken just before elections to actual results show a bias towards the pro-independence camp of around 3-4%, due probably to quieter majority of anti-independence supporters not shouting about their cause. Nobody expected SNP to lose their majority in 2016 due to this bias, so I bet £20 @ 10-1 that they would and I was £200 better off for this.

    It’s also suspicious that the Tories are polling at just 19% support having won 22% at 2016 HR election. I haven’t been aware of reduction of support for us on the ground (admittedly, I am very biased though).

    My guess is that this poll does not reflect opinion on the ground.

    BTW, I would never recommend banning opinion polls. They are part of the democratic process even if they do cheer up the Nats.

    Liked by 2 people

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s