We are changing the way we count people in hospital with confirmed COVID-19 as of 15 September 2020. The new figures are lower than the previous published management information and we have a break in the time series. This blog explains why and how.

Current situation

Every day since mid-March, the Scottish Government has been collecting data on the number of confirmed COVID-19 patients in hospital from across NHS Scotland. This data collection was set up quickly to meet an immediate need to understand the impact this new illness was having on our NHS hospitals. It was important to take a pragmatic approach to this in order to avoid any additional work for hospital staff, so while an approach was defined, local areas adopted the best way to extract this from their information systems.

The measure showed a rapid increase in the number of confirmed COVID-19 patients in hospital to over 1,500 in mid-April, reflecting the situation at the time. However, the measure stabilised at around 250 into July and August. Data published daily on coronavirus.data.gov.uk shows that on 26 August, for example, there were 442 COVID-19 patients reported in hospitals in England, 48 in Wales and 17 in Northern Ireland, compared to 249 in Scotland.

In addition, data from Public Health Scotland shows that the numbers of new hospital admissions where COVID-19 was confirmed were relatively low through July and August. In early July, additional information on hospital onset
cases
became available, and it was apparent that there were different types of inpatients being included in the figures provided each day by NHS boards, including those who have tested positive for COVID-19 at some point but are no longer being treated for COVID-19.

Investigation

On 26 August, we carried out a snapshot clinical audit to find out more about the relative contribution to the numbers of these different categories of patients (see below). In addition to using the available administrative information, this work involved clinicians in order to fully understand the status of current inpatients. The audit confirmed that the majority of COVID-19 inpatients reported in the daily published statistics were in hospital for an ongoing condition following a resolved COVID-19 infection, or for a condition unrelated to COVID-19. The majority of these were in the two largest Health Boards (Greater Glasgow & Clyde and Lothian), while some other Boards had previously adapted their own definitions to only include current cases, for example Fife did this in June resulting in a decrease in their figures.

Audit findings

The audit of inpatients identified 384 patients in hospital at 1am on 26 August across Scotland who had previously tested positive for COVID-19.

The majority of these patients (87%) were in hospital for a condition unrelated to COVID-19:

  • 45% were hospital onset cases who were no longer being isolated or treated for COVID-19;
  • 25% had recovered from COVID-19, been discharged and then readmitted for an unrelated condition;
  • 9% had a previous positive COVID-19 test in the community, and were admitted for an unrelated reason
  • 8% had been admitted for COVID-19, had recovered but were still in hospital for other reasons.

The audit showed that 8% of patients were either receiving treatment for COVID-19, were in rehabilitation after their treatment for COVID-19 was completed, or were in hospital for COVID-19 related complications. (5% of patients could not be classified into the above categories.)

What’s changing?

In order to make sure we are not counting people who are no longer being treated for COVID-19, we are introducing standard time periods after testing positive for hospital inpatients to remain included in the figures. Only patients who first test positive in their current hospital admission (or in the 2 weeks before admission) will be included. And they will stop being counted after 28 days in hospital (or after testing positive if that is later).

Because this is essentially a statistical rather than clinical definition, there will potentially be some patients who are still in hospital and ICU because of COVID-19 who will no longer be counted after 28 days. But overall, by making sure we are excluding those inpatients with a previous positive COVID-19 test who are now in hospital and ICU for unrelated reasons, the information will be more representative of the current situation in hospitals and more sensitive to any changes caused by new cases. We will keep this under review.

Comparing the initial data, it’s clear to see the impact of the change. The total number of confirmed COVID-19 inpatients under the previous definition was 262 on Tuesday 15th September while the new definition included 48 patients. Of these, 7 were in ICU on the old basis, and 6 under the new definition. We will no longer collect the data on the original basis.

Defining the measures in this way means that the data can be extracted from local information systems without requiring clinical input. This is a pragmatic solution that will ensure we are better placed to capture the impact of the evolution of the pandemic.

by Roger Halliday, Chief Statistician

I recently participated in the Administrative Data Research (ADR) Scotland public panel to discuss the development of Research Data Scotland (RDS) and focus specifically on the principles that underpin the new service.

 

The ADR Scotland public panel was created as a vital forum for understanding the views and perspectives of the public about the use of administrative data and to help to ensure research maximises public benefit in order to improve policies, services and, ultimately, lives. The panel consists of members of the public from across Scotland, from a range of different backgrounds and lived experience.

 

Our starting point for the discussion was the set of principles that underpin RDS’s overall mission to improve the economic, social and environmental wellbeing in Scotland by enabling access to, and linkage of, data about people, places and businesses for research in the public good. These principles are;

 

 

We had an engaging discussion about how ‘public good’ is defined. I explained that RDS is building upon the existing process where public benefit and privacy panels scrutinise data requests to assess their public benefit. Users of the RDS service will still need to make the case about how their proposed research project will improve wellbeing and reduce inequalities across Scotland.

 

I also provided a short update on our ongoing engagement with data controllers and progress against securing a range of data holdings. As part of the ADR Scotland partnership, operating within ADR UK and funded by the Economic and Social Research Council (ESRC), we have been working with data controllers across Scotland to secure a range of data holdings that are broadly aligned with Scotland’s National Outcomes.

 

We discussed how datasets held by RDS would be kept relevant for the future in light of, for example, socioeconomic movements. RDS will never hold onto data unless there is a clear purpose for doing so. However, there can also be exceptions in certain situations. As a result of this conversation, I have now spoken to colleagues at the SAIL databank about their data retention policies. I’m going to reflect upon their advice to develop a data retention arrangement for RDS which includes the safeguards to ensure we go about holding onto data in a safe and transparent way. My intention is to join a future meeting of the public panel to provide an update on this, as well as the other areas discussed, and seek any further feedback.

 

One area that generated more detailed discussion was around the topic of working with the private sector and how RDS intends to work with private companies who may want to request access to data.

 

I highlighted that RDS will carefully review all data access requests to confirm that the requestor holds the appropriate level of accreditation, ensure that the research is in the public good and in line with the data controller’s recognised use of their data. The first thing that any private company would have to do would be to partner with someone in the academic or public sector. Any data leaving the data safe haven would always be at an aggregate level and all research outputs will be checked to ensure they do not allow people or organisations to be identified.

 

To further support transparency, RDS will also develop an open register that captures uses of the data, details of who has accessed it, when and what happened to it. A condition of a researcher getting access to data would also be an agreement to publish the results. It is clear this is an area that needs careful consideration with appropriate measures put in place and this is something we will focus as we continue to develop the service.

 

I’m very grateful to the public panel for their time and comments and would welcome further discussion as we continue to develop RDS, so please get in touch via the RDS website if you’d like to be involved.

by Roger Halliday, Chief Statistician

For Scotland to continue to develop an approach to minimising the health, social and economic risks from COVID-19, we need robust and responsive evidence to make informed, real-time decisions in relation to the pandemic and to learn as we go.

 

For this reason, we have set-up the Scottish COVID-19 Data and Intelligence Network to build on the existing community of data expertise and make best use of the data already available across Scotland. The network is a partnership of expertise from across Local Authorities, Health Boards, Directors of Public Health, Health and Social Care Partnerships, Public Health Scotland, Scottish Government, academia and other public bodies.

 

The network aims to identify, prioritise, and develop data and intelligence products that address some of the key challenges relating to COVID-19 and ultimately help to improve lives.

 

The network has already delivered:

 

 

The next steps have been to focus on specific data challenges that include:

 

 

Throughout, the network will continue to follow the safe and secure arrangements established in Scotland for the use of data to ensure privacy and ethical data use is maintained. This includes ensuring the network’s activity is underpinned by a valid legal basis, using the Public Benefit and Privacy Panels to scrutinise the use of data and working with data infrastructure delivery partners who will have the highest level of UK accreditation standards. We are working to develop approaches enhancing these processes so they provide the same level of rigour, and enable data sharing to happen faster and more consistently.

 

As Scotland continues to drive down the number of COVID-19 cases and new challenges arise over the longer term, we will continue to develop a sustainable network to meet the public sector’s information needs. I’d welcome any feedback on the products we have developed or new issues that you would like us to address. Similarly, if you are a data owner or have data expertise that you think could be useful, then please get in touch.

Blog Updated on 28/07/2020

We have been publishing data every day since early April on the Scottish Government website. Given where Scotland is now in the pandemic, we think it’s a good time to refocus this publication.

Some of the information developments in Scotland include:

Public Health Scotland COVID-19 data – the weekly report published every Wednesday includes the latest data and trends on cases, hospital and ICU admissions, patient characteristics and Test & Protect. There is also a cases dashboard which is being developed further to include testing information and local area time series, as well as a wider health harms dashboard.

Scottish Government COVID-19 Four Harms Dashboardupdated every Monday at 14:00, this brings together data and evidence on the broader impacts of COVID-19. These are referred to as the four harms of COVID-19:

Work is also underway on a new public facing resource that will include interactive maps of COVID-19 cases at local authority level. We are working with Public Health Scotland to release this in July with plans to add further data in future.

The table below shows our planned changes for Wednesday 22/07/2020, but we will continue to review the site as things develop.

Web page section

Changes planned for Wednesday 22nd July

Overall layout

We will rearrange the layout of the page from a topic basis to a daily, weekly and since the start basis so people can see the latest figures first. We will include key weekly figures on hospital and ICU admissions from PHS and deaths from NRS.

Scottish numbers

 

These will be updated daily – and will likely include key information on testing, cases, deaths, care homes and hospitals.

We will continue to work with partners to improve the quality of data and will publish the total number of confirmed cases in ICU for Scotland but will no longer include suspected cases, as these fluctuate due to patient screening.

We will add in the total number of new tests reported (through NHS Scotland and UK labs).

We will include suspected COVID-19 cases in adult care homes pending a review of this data (see below).

Data by health board

These will be updated daily and we will keep the ongoing presentation and inclusion of this table on the webpage under review as other data and dashboards are developed.

Testing

Due to the stage we are at in the pandemic, with testing for all those aged 5 and over who are symptomatic, we will no longer collect data on testing of key workers and their symptomatic household members, which commenced to allow key workers to return to work promptly. We now report on specific key workers such as care home staff.

We will continue to report the numbers of tests daily, and to date and will add a weekly figure.

Care homes

We will keep publishing the daily and cumulative number of suspected COVID-19 cases in adult care homes pending a review of the data given current low levels and caveats around the data as well as the new information that is emerging through Test & Protect.

We will update the number of adult care homes with active cases weekly rather than daily.

Blog Updated on 28/07/2020 Extensive testing is routinely being carried out in care homes, enabling the identification of positive confirmed cases quickly via the Public Health Scotland (PHS) eCOSS system. The usefulness of suspected cases at this stage in the pandemic is therefore reduced. To ensure we continue to adequately monitor the situation in care homes and to provide public confidence, we will discontinue the publication of suspected cases from 29 July and replace with the number of confirmed cases among care homes residents on a weekly basis using eCOSS data. The first weekly data would be published from 5 August.

Community health & care

We will no longer report NHS24 calls, Coronavirus Helpline calls and ambulance attendances and conveyances.  We will instead refer to the PHS weekly report.

We will continue to update information on numbers of people advised to shield and accessing support on a weekly basis, and will shortly also begin reporting on deaths in the shielding group (Thursdays).

We will update the delayed discharge figure once a week (Thursdays) rather than every weekday.

Health & Social Care Staff

We will change to weekly rather than daily updates for NHS staff absence (Wednesdays).

Excel downloads

We will leave all data previously published as a record for information. We will stop publishing new data on suspected cases in hospital and ICU as well as those mentioned above. We will add explanatory notes to the data files.

Where we have changed from daily to weekly, where possible we will add a weekly series back in time and convert the chart to a weekly chart.

Care homes page

We will no longer publish additional breakdowns on suspected COVID-19 cases in adult care homes.

We will continue to publish weekly adult care home testing data and COVID-19 deaths reported to the Care Inspectorate.

Current analysis and data around COVID-19

Scottish Government (SG), Public Health Scotland (PHS) and National Records of Scotland (NRS) have been working together to produce a range of information to provide insight into the COVID-19 pandemic and the effect it is having in Scotland.

Scottish Government publish daily coronavirus data for Scotland (updated each day at 2pm) that shows number of new confirmed COVID-19 cases, new reported deaths of people who have tested positive, number of tests carried out (and whether results were positive or negative), and numbers of people in hospital. It also show the number of NHS24 calls, Scottish Ambulance Service Attendances, the number of people delayed in hospital, and numbers of suspected COVID-19 cases in Care Homes.

National Records of Scotland are publishing weekly National Statistics on deaths involving coronavirus, and Public Health Scotland are releasing weekly data on the cumulative number of positive COVID-19 cases by age, sex and deprivation, admissions to ICU, daily number of COVID-19 related calls made to NHS24 and daily number of consultations with COVID-19 Community Hubs and Assessment Centres. PHS also produces daily and weekly open data, a daily cases and deaths (including excess deaths) dashboard, a weekly wider impacts dashboard, and has plans to publish different topic area COVID-19 reports.

Public Health Scotland’s COVID-19 publications are accessible here: https://beta.isdscotland.org/covid-19-and-the-production-of-statistics/  and all their forthcoming publications are accessible here: https://beta.isdscotland.org/forthcoming-publications/.

Upcoming releases of analysis and data around COVID-19

During this public health pandemic, in line with guidance from the Office for Statistics Regulation, we are planning to publish a range of additional analysis on topics such as numbers of people identified as at high clinical risk of COVID-19 and advised to shield, numbers of tests carried out, causes of death, and other analysis of demographic characteristics. We know that there is a lot of demand for this data, and doing so will make sure that information relevant to the COVID-19 crisis is openly available to inform the ongoing public debate.

Some of the data being reported is management information (data that is gathered routinely by public bodies to inform their day to day operations). Management information produced by the Scottish Government is subject to voluntary application of the Code of Practice for Statistics, and our aim is to make sure it is published in a way that maintains trustworthiness, quality and value to users. For example, difficult choices may need to be made on whether data are of sufficient quality to support the use being made, and the professional analysts producing the analysis, will balance these risks to quality with making sure that they are published as quickly as possible.

When management information is used publicly to inform Parliament, the media and the public, it should be published in an accessible form, with appropriate explanations of context and sources.  It is important therefore that this information is released in a transparent way to ensure that it is equally available to all. As part of that transparency, we preannounce our plans for what management information is going to be published, and when, noting that these analyses are often complex and involve multiple data sources, and plans may by necessity be updated at short notice.

Our current plans for release of new information are:

Open data around COVID-19

Scottish Government, National Records of Scotland, and Public Health Scotland are all publishing open datasets on the COVID-19 pandemic for people to use and reuse. This helpful blog post gives more details of what is available and where to access these sources of open data on covid-19.

The Office for National Statistics have also published their plans for further analysis of COVID-19 data.

by Roger Halliday, Chief Statistician

I wanted to share an update on how Research Data Scotland is supporting the national response to COVID-19.

 

RDS’s mission remains to improve the economic, social and environmental wellbeing in Scotland by enabling access to and linkage of data about people, places and businesses for research in the public good.

 

In response to the national emergency, a new data taskforce, of which I am chair, has been established. While Scotland have been delivering high quality research in a secure and ethical way for a number of years, the ambition of the data taskforce is to enable evidence-based policy and operational decisions in response to the current COVID-19 situation.

 

This will be achieved by building on and repurposing our existing and emerging data infrastructure which includes resources, expertise and capabilities offered by service delivery partners and partner organisations, including The Scottish Government, PHS (eDRIS), NRS, EPCC and HDR-UK, alongside representatives from the University of Aberdeen, University of Dundee, University of Edinburgh, University of Glasgow, University of St Andrews and University of Strathclyde. This builds on existing, accredited infrastructure and methodologies that enable us to maintain privacy and security while delivering outputs quickly.

 

Researchers will be able to carry out collaborative research and analysis that provides evidence for important COVID-19 related decisions, initially to support decisions on lifting social restrictions.

 

This group has identified key datasets that will help research to support decisions on lifting social restrictions, and the development of a plan for securing these datasets. This includes data on;

 

 

An infrastructure has also been established to hold and link data securely, and to provide secure online access for collaborative research. This has been developed by the Public Health Scotland and Edinburgh Parallel Computing Centre, and offers a virtual analytical workbench to allow research teams to work on the same datasets from different locations.

 

Work is well developed to secure these datasets, meanwhile forthcoming emergency data sharing legislation will allow us to operate quickly, while ensuring that the data is accessed legally and ethically. We’d also welcome suggestions for additional datasets that may support emerging sets of needs.

 

For more information about accessing this data please visit ResearchData.Scot

By Roger Halliday, Chief Statistician

 

The use of data has greatly assisted us in understanding COVID-19 and will help our response to the pandemic.  The use of data for this type of research is very important and gaining access to the appropriate data remains both legal and ethical.

 

To ensure that we balance our legal and ethical obligations whilst providing data for research in a timely manner, we have temporarily revised our process to access data for projects that will assist with the COVID-19 response.

 

The revised process can be found in this statement and details of the projects being approved using this process are available on request.

Scottish Government (SG), National Records of Scotland (NRS) and Public Health Scotland (PHS) are all publishing open datasets on the COVID-19 emergency for people to use and reuse.  We know that there’s lots of interest in open data on COVID-19 with people keen to use it in briefings, data journalism, documents, apps and dashboards.   It’s really encouraging to see such an engaged and lively open data community response and so the purpose of this blog is to help signpost users to relevant open datasets produced by these organisations.  We intend to update this blog as more relevant datasets become available.  Of course, given all of this interest, it is important to have accurate and robust information made available as open data for reuse and for decision making and the teams in SG, NRS and PHS are working hard to uphold these standards.

It is strongly advised that people using these data use the published metadata which are produced to support each dataset. The metadata describe the data and provide more information about it.    This is vital for the effective use, reuse and interpretation of datasets, and any products that are created using these data.

We welcome feedback on the data and how you’re using it.  Whilst our team cannot respond to detailed questions on the data itself, we’re happy to pass this on to the teams who produce it.

SOURCES OF OPEN DATA

The Scottish Government is publishing daily management information  on a number of COVID-19 indicators on Scotland’s statistics open data platform, www.statistics.gov.scot  as 5* linked open data.

http://statistics.gov.scot/data/coronavirus-covid-19-management-information.  Data are available at Scotland level; some data are also available at Health Board level.    We’d like to thank our contractors, Swirrl, for their help in supporting us to get these datasets published.

Our contractors at Swirrl have recently published a series of short videos to help users to explore the 5* linked open datasets on www.statistics.gov.scot: https://guides.statistics.gov.scot/article/53-video-guides

We have published a series of background helpguides for programmers to build apps and dashboards from www.statistics.gov.scot: https://guides.statistics.gov.scot/category/37-api

Our contractors at Swirrl have also published a blog on the reuse of the daily COVID-19 management information:  https://medium.com/@northernjamie/covid-19-in-scotland-four-ways-to-get-the-data-e12c1b450a5.  Their blog provides details on how to reuse the daily data from www.statistics.gov.scot, using 4 different packages – using R, Python, Google Sheets, and Tableau.  There are examples of code which can be used to generate apps, dashboards and visualisations for each of the packages, as illustration of how the data can be retrieved.   At the end of the blog, there is a section on the importance and benefits of publishing open data.

We have also published the R code used to generate these datasets as open data, as well as the 3* csv files on the DataScienceScotland GitHub repository : https://github.com/DataScienceScotland/COVID-19-Management-Information .  We have done this to help users in the open data community who we know are building and updating apps and dashboards from these data.

Whilst we intend to publish the daily management information COVID-19 data by mid-afternoon each day,  there may be, at times, unforeseen delays in getting the daily open data published.  Please note that if this is the case and you need the data urgently, published data can usually be found in Excel format at 2pm on the Scottish Government website.  We thank you in advance for your patience.

 

National Records of Scotland (NRS) are publishing weekly National Statistics on www.statistics.gov.scot on deaths involving coronavirus (COVID-19) http://statistics.gov.scot/data/deaths-involving-coronavirus-covid-19   .Data are available at Scotland level, and for some measures at council area and health board level.

This supports the weekly NRS publication:  https://www.nrscotland.gov.uk/covid19stats .  Further information and other relevant demographic data can be found within the NRS blog “Statistics relevant to COVID-19”: https://blog.nrscotland.gov.uk/2020/04/23/statistics-relevant-to-covid-19/

 

Public Health Scotland publishes daily and weekly open datasets on the Scottish health and social care open data platform as 3* open data files. Data are available on the daily and cumulative number of positive cases of novel coronavirus (COVID-19) in Scotland, including cumulative number of positive cases at health board and council area level, and deaths at Scotland level. Data are released weekly on the cumulative number of cases by age, sex and deprivation, admissions to ICU, daily number of COVID-19 related calls made to NHS24 and daily number of consultations with COVID-19 Community Hubs and Assessment Centres.

It is intended that the scope of COVID-19 related data within this open data platform will increase, as more evidence becomes available. The COVID-19 section of the portal can be accessed at: https://www.opendata.nhs.scot/dataset/covid-19-in-scotland  and https://www.opendata.nhs.scot/dataset/weekly-covid-19-statistical-data-in-scotland

 

 

BACKGROUND AND USE CASES

Open data are those that are accessible, free of restriction on use or redistribution in its licensing conditions and in a digital, machine readable format for use with other data.  They are published according to the 5* deployment scheme for open data.

The datasets which are referred to in this article are all published under the Open Government license.  Given that these data are open, they are intended for anyone to use and build apps and dashboards from.

Below is an example of an official dashboard – the COVID-19 Scotland dashboard, produced by Health Protection Scotland and Public Health Scotland: https://www.arcgis.com/apps/opsdashboard/index.html#/658feae0ab1d432f9fdb53aa082e4130

We are also aware of some good examples of other dashboards which have been published using open data.    These have been created with the support of some of our Open Government civil society partners and the open data community more generally.

This is a good example of what someone using open data can build with it. https://smazeri.shinyapps.io/Covid19_Scotland/ .  This dashboard has been produced with support from the Epidemiology, Economics and Risk Assessment Group (EERA) within the Roslin Institute.  It is important to remind people looking at public data and dashboards produced by individuals  that different countries and organisations may gather data differently using different tests and definitions so we need to be careful about direct comparison, and methodologies can change over time.

 

Scotland is a member of the Open Government Partnership: https://www.gov.scot/policies/improving-public-services/open-government-partnership/ .  In 2019, Scotland’s second Open Government Action Plan was launched in conjunction between Scottish Government with civil society representatives, and includes a commitment to improve how information and data are shared: https://www.gov.scot/publications/scotlands-open-government-action-plan-2018-20-detailed-commitments/pages/4/.  This work has progressed with the help and input of our partners in the Open Government Network.  We would like to thank our Open Government civil society partners and the open data community more widely for their continued support and enthusiasm in highlighting and promoting the value of using and reusing open data.

Rural and Environmental Science and Agricultural Services (RESAS) collect a range of statistical information on the agricultural sector which is the evidence base for the majority of our analysis on agriculture. However, in light of the current guidance from the Scottish Government and the Chief Medical Officer we have had to change our planned schedule of statistical surveys in agriculture for 2020.

These changes have been developed in consultation with our colleagues in DEFRA and the other devolved administrations. While Wales and Northern Ireland are still assessing the options for their June Census collection, we have all agreed to take a common approach to ensure UK figures can still be produced on all major agricultural surveys.

This approach ensures we comply with current legislation and guidance provided by both the Government Statistical Service and the UK Statistics Authority. The approach being taken is also in line with guidance being issued on all major and face-to-face surveys conducted by the Scottish Government.

For a full explanation of the changes to our proposed schedule for 2020, please see the attached letter to stakeholders..

In light of the extraordinary situation we are now in, in relation to COVID-19 and given the challenges we are all facing, this is an update on our contingency arrangements. As our analytical staff are supporting mission-critical work with other colleagues across government, we have had to make some difficult decisions in relation to current data access arrangements.

Due to the current increased pressures, we are not accepting any new bespoke data access requests for Scottish Government datasets or NRS census data, unless directly COVID-19 related. This will include variations (amendments) to existing data access permissions which have already been granted. We will review the situation over the course of the current public health emergency in light of Government advice and public health guidance.

We want to continue to support research, and those bespoke requests we received before 5pm on 24th of March will be reviewed and considered as normal, although please bear with us, as it may take longer than usual to reply to you. If your project is being handled by eDRIS, please also contact your research co-ordinator and refer to the eDRIS service users’ update of 19 March 2020.

Our data is still accessible via the Open Data Platform: www.statistics.gov.scot or in over the one hundred bulletins we publish each year, most of which include data that can be downloaded.

For further details please see our full 24 March statement.