Statistics

Working as a government statistician – Maike Waldmann

March 8, 2023 by Helen Davidson No Comments | Category Working in statistics

This article has been produced as part of a blog series to spotlight the work of analysts and statisticians across government. In this blog, hear from Maike Waldmann about her work as part of the Poverty and Deprivation Statistics Team.

The poverty and deprivation statistics team within the Scottish Government is a team of two, my team leader (C1 statistician) and me (B3 Senior Assistant Statistician). The team is responsible for creating Official Statistics on household income, wealth, poverty, and deprivation including the Scottish Index of Multiple Deprivation. In addition to analysing the data and writing reports, we also support ministers and the general public with any queries on these topics.

I’ve been in my current post for five years, and I would call it a traditional stats post. Among other things, I’m responsible for publishing reports on household income and poverty every year, and on personal wealth and debt every two years. This involves: 

• in-depth knowledge of several large population surveys that focus on household finances 
• data analysis and statistics 
• explaining what the numbers mean 
• producing ad-hoc analysis related to household finances for colleagues within the Scottish Government or external organisations 

Statistical reporting

Over the course of my time in post, I have slowly changed the way we create our statistical reports, automating most steps from receiving the data to creating the report documents. In government, we call this a ‘Reproducible Analytical Pipeline’, or RAP in short. When I started in this post, the reports used to be pdf documents that I created via SAS statistical software, and MS Excel and Word, with lots of copying and pasting and linked spreadsheets, and many manual steps. Now, I have automated almost everything: the data analysis, creating charts and spreadsheets and most of the reports as well. This results in accessible interactive html reports that are a lot easier to browse through and engage with. They are also easier and quicker for me to produce. I can spend more of my time understanding and explaining what the numbers and trends mean.

Benefits of a Reproducible Analytical Pipeline

There are obviously many benefits to automation, mainly to do with efficiency. RAP is widely promoted within the UK government statisticians group. But it’s also a lot of fun to automate things and see repetitive tasks disappear, to be replaced with increasing coding skills.

Every year I automated another part of the reports. When I was new in post, I started off knowing just the basics of the coding language R, but over time, I learned a lot on the job. One year, I created the charts for my poverty reports in R, replacing the charts made with Excel. I then no longer needed to format each chart individually. Another year, I automated how we made the accompanying spreadsheets, a major time saver. For this, I also needed to learn some VBA (the language used to write Excel macros), which can be integrated into the R workflow. I also built the html reports as a standalone website using R Markdown. I learned about web accessibility in order to make the content easily available to any user. Building the website required me to dig deeper into html, CSS and JavaScript code, as well as learning about good design. We now publish our reports as ‘html first’, as opposed to ‘pdf first’.

Automating the poverty reports has taken a few years with small changes each year, but now the whole automated process from data import to (almost) final report output takes minutes. I can spend more time on the wording – pulling out the key messages clearly and concisely, while also offering all necessary caveats. This benefits our team a lot – we often reuse paragraphs word-for-word from the report when briefing ministers, or responding to requests, and are able to respond quickly to many stats questions.

So while I have been in the same post for some time, publishing the same reports every year, it hasn’t been boring at all. I have had a lot of fun doing this and learned loads, while improving the outputs as well as the processes to get there.

Latest poverty report

You can find the latest Poverty and Income Inequality in Scotland 2018-21- analytical report on the Scottish Government website. 

Tags: data, Scotland, Scottish Government, statistics

Comments