Statistics

Sources of open data on COVID-19

May 11, 2020 by martinmacfie No Comments | Category Uncategorized

Scottish Government (SG), National Records of Scotland (NRS) and Public Health Scotland (PHS) are all publishing open datasets on the COVID-19 emergency for people to use and reuse.  We know that there’s lots of interest in open data on COVID-19 with people keen to use it in briefings, data journalism, documents, apps and dashboards.   It’s really encouraging to see such an engaged and lively open data community response and so the purpose of this blog is to help signpost users to relevant open datasets produced by these organisations.  We intend to update this blog as more relevant datasets become available.  Of course, given all of this interest, it is important to have accurate and robust information made available as open data for reuse and for decision making and the teams in SG, NRS and PHS are working hard to uphold these standards.

It is strongly advised that people using these data use the published metadata which are produced to support each dataset. The metadata describe the data and provide more information about it.    This is vital for the effective use, reuse and interpretation of datasets, and any products that are created using these data.

We welcome feedback on the data and how you’re using it.  Whilst our team cannot respond to detailed questions on the data itself, we’re happy to pass this on to the teams who produce it.

SOURCES OF OPEN DATA

The Scottish Government is publishing daily management information  on a number of COVID-19 indicators on Scotland’s statistics open data platform, www.statistics.gov.scot  as 5* linked open data.

http://statistics.gov.scot/data/coronavirus-covid-19-management-information.  Data are available at Scotland level; some data are also available at Health Board level.    We’d like to thank our contractors, Swirrl, for their help in supporting us to get these datasets published.

Our contractors at Swirrl have recently published a series of short videos to help users to explore the 5* linked open datasets on www.statistics.gov.scot: https://guides.statistics.gov.scot/article/53-video-guides

We have published a series of background helpguides for programmers to build apps and dashboards from www.statistics.gov.scot: https://guides.statistics.gov.scot/category/37-api

Our contractors at Swirrl have also published a blog on the reuse of the daily COVID-19 management information:  https://medium.com/@northernjamie/covid-19-in-scotland-four-ways-to-get-the-data-e12c1b450a5.  Their blog provides details on how to reuse the daily data from www.statistics.gov.scot, using 4 different packages – using R, Python, Google Sheets, and Tableau.  There are examples of code which can be used to generate apps, dashboards and visualisations for each of the packages, as illustration of how the data can be retrieved.   At the end of the blog, there is a section on the importance and benefits of publishing open data.

We have also published the R code used to generate these datasets as open data, as well as the 3* csv files on the DataScienceScotland GitHub repository : https://github.com/DataScienceScotland/COVID-19-Management-Information .  We have done this to help users in the open data community who we know are building and updating apps and dashboards from these data.

Whilst we intend to publish the daily management information COVID-19 data by mid-afternoon each day,  there may be, at times, unforeseen delays in getting the daily open data published.  Please note that if this is the case and you need the data urgently, published data can usually be found in Excel format at 2pm on the Scottish Government website.  We thank you in advance for your patience.

National Records of Scotland (NRS) are publishing weekly National Statistics on www.statistics.gov.scot on deaths involving coronavirus (COVID-19) http://statistics.gov.scot/data/deaths-involving-coronavirus-covid-19   .Data are available at Scotland level, and for some measures at council area and health board level.

This supports the weekly NRS publication:  https://www.nrscotland.gov.uk/covid19stats .  Further information and other relevant demographic data can be found within the NRS blog “Statistics relevant to COVID-19”: https://blog.nrscotland.gov.uk/2020/04/23/statistics-relevant-to-covid-19/

Public Health Scotland publishes daily and weekly open datasets on the Scottish health and social care open data platform as 3* open data files. Data are available on the daily and cumulative number of positive cases of novel coronavirus (COVID-19) in Scotland, including cumulative number of positive cases at health board and council area level, and deaths at Scotland level. Data are released weekly on the cumulative number of cases by age, sex and deprivation, admissions to ICU, daily number of COVID-19 related calls made to NHS24 and daily number of consultations with COVID-19 Community Hubs and Assessment Centres.

It is intended that the scope of COVID-19 related data within this open data platform will increase, as more evidence becomes available. The COVID-19 section of the portal can be accessed at: https://www.opendata.nhs.scot/dataset/covid-19-in-scotland  and https://www.opendata.nhs.scot/dataset/weekly-covid-19-statistical-data-in-scotland

BACKGROUND AND USE CASES

Open data are those that are accessible, free of restriction on use or redistribution in its licensing conditions and in a digital, machine readable format for use with other data.  They are published according to the 5* deployment scheme for open data.

The datasets which are referred to in this article are all published under the Open Government license.  Given that these data are open, they are intended for anyone to use and build apps and dashboards from.

Below is an example of an official dashboard – the COVID-19 Scotland dashboard, produced by Health Protection Scotland and Public Health Scotland: https://www.arcgis.com/apps/opsdashboard/index.html#/658feae0ab1d432f9fdb53aa082e4130

We are also aware of some good examples of other dashboards which have been published using open data.    These have been created with the support of some of our Open Government civil society partners and the open data community more generally.

This is a good example of what someone using open data can build with it. https://smazeri.shinyapps.io/Covid19_Scotland/ .  This dashboard has been produced with support from the Epidemiology, Economics and Risk Assessment Group (EERA) within the Roslin Institute.  It is important to remind people looking at public data and dashboards produced by individuals  that different countries and organisations may gather data differently using different tests and definitions so we need to be careful about direct comparison, and methodologies can change over time.

Scotland is a member of the Open Government Partnership: https://www.gov.scot/policies/improving-public-services/open-government-partnership/ .  In 2019, Scotland’s second Open Government Action Plan was launched in conjunction between Scottish Government with civil society representatives, and includes a commitment to improve how information and data are shared: https://www.gov.scot/publications/scotlands-open-government-action-plan-2018-20-detailed-commitments/pages/4/.  This work has progressed with the help and input of our partners in the Open Government Network.  We would like to thank our Open Government civil society partners and the open data community more widely for their continued support and enthusiasm in highlighting and promoting the value of using and reusing open data.

Tags: COVID-19, open data, Open Government, statistics

Comments