Statistics

Statement from the Chief Statistician on the Annual Population Survey

November 14, 2024 by lisaclark No Comments | Category Economy, Statistics Policy

Challenges in Gathering Scotland’s Data: How Response Rates Are Impacting the Annual Population Survey

In recent years we have seen difficulties in getting people to respond to surveys and other data collections. These problems are affecting most surveys across the UK. In this blog I will describe the problems with the Annual Population Survey and how it is affecting the ability to provide robust statistics on Scotland’s Economy, including the labour market. 

The Annual Population Survey 

The Annual Population Survey (APS) is a UK wide survey conducted by the Office for National Statistics (ONS). The APS combines results from the ONS Labour Force Survey (LFS) with local LFS boosts. Scottish Government funds the Scottish boosted sample of the LFS to create the APS, which we can use to produce estimates for local authorities and smaller groups of the population in Scotland. 

However over the last several years, and particularly since the pandemic, sample sizes for the APS and LFS have fallen dramatically. This has led to difficulties in presenting the data from the surveys and reduced our confidence in the estimates that the APS data helps us to provide statistics on. This has an impact on what data we can report. 

This is impacting the whole of the UK. The Chief Economist from the Bank of England recently wrote to ONS highlighting that the falling sample sizes and uncertainty in the LFS and APS data is impacting on the central bank’s decisions on interest rates.  

In response to this letter and other criticisms, ONS conducted analysis to assess the impact. They concluded that the APS estimates are robust at National and headline regional level (which includes Scotland). But because of this analysis, and other issues, including not yet being able to take into account the new population statistics from the UK Censuses, the ONS wrote to the statistical regulator about the quality concerns, and the OSR decided to remove their statistical accreditations on these statistics. 

The ONS recently stated that it is still appropriate to use the estimates from the data. But users should be aware of these issues, in particular, the uncertainty around these estimates in recent years. 

Why the APS matters to Scotland 

The APS is an important source for us to produce statistics and describe the Scottish Economy. It helps us to understand the performance of the Scottish and local economies, covering key metrics such as employment, unemployment and economic inactivity. 

They are also used to inform policy decisions, such as the commitments within the Scottish Government’s Programme for Government on Economic Growth and Child Poverty. But these statistics are also a vital component of measuring progress and helping local authorities tailor policy decisions to meet local need.  

This is a fact the Office for Statistics Regulation picked up in their response to ONS and stated “ONS must ensure that its analysis of the quality of APS-derived statistics is sufficient to enable the Scottish Government and Welsh Government, and other producers of APS-derived statistics, to reach a view on the quality of their APS-derived statistics.” 

Our Commitment to providing Economic Statistics 

We are working with Welsh and Northern Irish Governments and our colleagues in ONS to better understand the quality issues and express how these issues are affecting the ability for our statisticians to produce quality statistics on the economy that meets users’ needs. ONS are taking these concerns seriously and are working with us to understand how the quality issues are affecting results for Scotland. 

We are also exploring whether alternative data sources exist that could supplement the LFS/APS data for Scotland, including how we better use administrative data, and considering how and for what purposes we best use the LFS and APS data going forward. We will also be continuing to caveat the headline labour market figures we publish to ensure our users are kept fully aware of the uncertainty around even the Scotland level data.  

What next and where to get more information 

We may need to make some difficult decisions. For example, to ensure we, and our users, have full confidence in the data we publish, we may need to pause or delay some of our regular Labour Market publications to allow for additional quality assessments of the data prior to publication. 

While this is not a position we want to be in, it is vital we create the time and space to ensure our publications continue to provide the best value and quality data possible for our users. We will keep any disruption to publications to a minimum and keep users informed of changes on the Scottish Government’s Labour Market Statistics webpage as well as via ScotStat emails for those on the mailing list. 

We will also be working with NOMIS who host these statistics for the UK and Scotland. They plan to continue providing sub-national APS data while we may pause our statistical publications. We will work with colleagues at NOMIS to make sure appropriate caveats are included where we have identified concerns on sub-national Scottish data. 

We also plan to publish our quality assessment by the end of 2024. 

If you want any further information please contact the Scottish Government Labour Market Statistics team: LMStats@gov.scot 

Tags: Annual Population Survey, APS, Chief Statistician, Economic Statistics, Scotland, statistics, Surveys

Comments