Notes on Statistics (updated January 2019)
1. THE ADJUSTMENT TO YOUGOV RESULTS (the maths of the adjustment can be found here)
The headline percentage difference provided by some of the polling organisations (particularly YouGov) have been adjusted, in this study, to take account of three things
The fact that some (particularly YouGov polls) only sample from Great Britain, not the UK (which includes Northern Ireland) plus Gibraltar. Both Northern Ireland and Gibraltar voted significantly for Remaining in the EU, so a (very small) factor needs to be added to the Remain estimate. The referendum Leave majority in GB alone was 4.21% and the majority in UK+Gib was 3.78%, so, using that as a basis, the figures require a GB to UK balancing factor of typically 0.43% added to the final Remain figure.
YouGov's weightings do not correspond with the referendum vote split. This is because they use a technique called rim-weighting to equalise, where possibe, their weights across a number of factors (including, for example, voting in the last election, age group, gender etc. YouGov overweight both leave and remain votes but they do so in proportion to the votes cast at the referendum. That has two effects, it slightly overweights Leave by typically 0.34% so 0.34% should be added to the Remain figures (see below) and, more significantly, it underweights the preferences of those who did not vote in the referendum but who now express a preference. Among that group the preferences are approximately 20% to Leave, 48% to Remain and 32% Don't know. There has been a consistent majority for Remain in that group. However that group is underweighted by about 10% by YouGov's choice of weights. This therefore requires, typically, a further 1.8% added to the final Remain majority. This significant addition is explained in more detail later below and a dashboard is provided for those who want to try different numbers.
YouGov quote three percentage headline figures: Remain, Leave and Don't know. At any referendum there are no 'don't knows'. However the 3.8% referendum majority for Leave was based on the difference only between 48.1% voting Remain and 51.9% voting Leave, ignoring any don't knows. So a YouGov result that gives, say, 45% Remain, 40% Leave and 15% don't know, on the face of it looks like a 5% majority for Remain. However, discarding the don't knows, the 45:40 would then be scaled up to 52.9% : 47.1% which makes the majority 5.8% rather than 5%. This adds a variable factor (in the example here 0.8%) to either Remain or Leave, depending on which is the largest.
TOTAL between 3% and 5% underreported for Remain
These minor factors all accumulate which is why the figures need adjustment. Each set of figures provides a different adjustment, and the adjustments are described below. But typically the adjustment is 3%-5% extra to be added to the Remain majority from the raw difference in the YouGov figures (though each poll gives a slightly different addition. Clearly this makes a headline difference because 3%-5% is around the Leave majority at the referendum, so any YouGov figure suggesting a majority for Leave, unless it is a Leave majority of at least 3% is still, actually, a national preference for Remain.
These processes are described in more detail below.
YouGov undertake their survey asking individuals what social group/voting group/age group etc., they belong to. They then weight the poll so that it corresponds to the national demographics. For example, if they asked 1000 people whether they supported the reintroduction of grammar schools, but the sample they asked turned out to be 100 Labour voters and 900 Conservative voters, they would have to divide the answers given by the Conservative voters by 9 if the national demographic were that the same number voted Labour as voted Conservative.
Polling companies publish their target weights (see section 2 below) but those target proportions for the remain/leave/did-not-vote do not always correspond to the actual UK+Gib proportions at the referendum. Partly this is due to (a) but that only has a minor effect. The adjustment here is to undo their weightings and then re-weight the polls so they exactly correspond to the proportions at the referendum.
HOW IS THIS DONE?
The weights in YouGov, for instance, under weigh the 'did-not-vote' group. Yougov uses 18%, the actual figure is about 28% non-voters. The calculation is simple.
The figures typically come in the structure shown. Figures in black are typically supplied by the poll. Figures in red are calculated from the black or from other figures in red. The bottom two rows, for example, simply apply the percentages given in the polls to the actual votes at the referendum. The polls do not give the 'did not vote' percentage, but that can be deduced from all the other figures in the poll, as shown below, and then applied to the number of electors who chose not to vote.
You can put your own figures in and see the outcome using the button to recalculate. Figures shown are as truncated integers but actual calculations use real numbers, so totals will not be exact due to rounding errors.
Select from the YouGov polls: or put in your own figures
*Note that the maths for this is on this page. This figure is directly comparable with the 3.8% Leave preference at the referendum
2. UNHELPFUL POLL WEIGHTINGS
YouGov, for instance, use the following poll target percentage weightings (see, for instance page 8 on April 13th poll):
EU referendum Vote (Election Result)
Target percentage weightings used in BMG UK poll Oct 2016
(only two polls in the study are from BMG)
Target percentage used in all YouGov weightings
(51 polls in study are from YouGov)
|Actual UK+Gib referendum votes||Actual UK+Gib percentage|
|Don't Know / Did Not Vote||27.2%||18.0%||12,913,178||27.8%|
BMG and YouGov figures are shown in blue, actual voting figures shown in black
It can be seen that YouGov target weighting percentages over-weight the voters (Remain and Leave rows) and underweight the 'did not vote/don't know' group. This has the effect of giving (a) an apparent (but not real) greater majority for Remain than is the case: 42.7% - 39.3% = 3.4% (or 4.14% just amongst voters) majority by comparison with 37.5% - 34.7% = 2.8% (or 3.8% when just amongst voters, which was the actual winning margin) and (b) given the 'don't know/did not vote' group has a strong majority (currently running at about 20%) in favour of Remain, YouGov will only be including 18% of that, and not 27.8%. i.e. they will not be including that contribution as much as it should be included, which diminishes the poll count for Remain. Overall the effect of these weights is to report a significantly diminished Remain majority.
The BMG weightings, by comparison (green highlights above) which are for the UK (but excluding Gibraltar), correspond much more closely to the actual referendum percentages. Note they do not add up to 100 because some will not have answered the question and they have been given a weighting in the BMG poll.
The adjustments made in the presentation of the results using the scheme in section 1 above, reverse that YouGov under-reporting. Averaged over the 30 polls, the under reporting is just under 2%, so, for example, where YouGov says that leave and remain are equal on 44%, the figures are more likely to be 43% in favour of leaving and 45% in favour of remaining. Obviously that would provide a significantly different referendum result.
A comparison of weightings is shown at the top of the page. The Yougov basic differences are shown as thin blue lines, about 2% lower, on average, than the adjusted data. The green line is a regression on the adjusted data and the black line a regression on the YouGov data. The green line is slightly steeper than the original and is shifted up by about 2%.
3. THE NORTHERN IRELAND and GIBRALTAR ELEMENT
The YouGov polls do not include Northern Ireland or Gibraltar in their polling. They voted to remain by about 120,000 (combined). The polls (mostly only GB), nevertheless, have been implements on the total UK number so the calculation will generally be an underestimate of the Remain vote. Whilst the majorities are small (of the order of 1%-2%), to do the calculation accurately would be within the noise of the poll, but a rough estimate could be made by, simply, adding 120,000 to the REMAIN majority which is approximately an extra 0.3%.
4. CLAIM THAT THE CURVE REPRESENTING THE LEAVE MAJORITY IS MARGINALLY INCREASING
The entry 'marginally increasing' is based on (a) the very good linear (and polynomial) fit to the two data sets (non voters from last time) on the third graph which implies that the majority is increasing and currently at about 18% (and increasing) whilst (b) the number of new 18 year old voters is about 750,000 per year, which a similar number of deaths, mostly over 65. The 18-24 group saw a majority for Remain of about 75% whilst the over 60's were about 35% for Remain. Assuming people stick to their votes (which was the case up until the May 2017 election but is no longer the case for Leave voters), then there is a new electorate of about 32,000 (=40% of 80,000) per month in favour of Remain. Over a year, given turnout rates etc., that corresponds to approximately 150,000 more remain votes and 250,000 less leave votes. That will mean a small but increasing majority for Remain. One study by the Financial Times puts the tipping point even if no abstainers vote, at the end of 2021. I think it much earlier than that, indeed we may have already passed that point, see here.
There is a dashboard in which you can test scenarios as to when the voters (ignoring - or not - the non-voters) would comprise new 18 year-olds who would, according to the polls, overturn the simple vote. Of course, if there were a referendum tomorrow, the polls comprehensively suggest that Remain would win by more than 1 million.
6. THE 12.7 MILLION
More Remainers than Brexiteers failed to vote on June 23rd 2016. That is almost certainly related to the pre-polling expectation that Remain would win, so there was more complacency amongst the remain voters. A similar effect may have put Trump in place in the USA and there are a number of studies which demonstrate that effect (see articles referred to the in main web page for references).