Digital

Care, code and text: Accelerating understanding of care data

March 28, 2025 by Stewart Hamilton No Comments | Category Data Science and Innovation Accelerator, Digital Scotland

Guest blog post by Sam Henderson-Palmer, Information Analyst, Care Inspectorate.

The Data Science and Innovation Accelerator is a programme for public sector organisations in Scotland designed to support the innovative use of data to solve real-world business problems or discover new opportunities. In his blog, Sam Henderson-Palmer, Information Analyst in the Risk and Intelligence team of the Care Inspectorate reflects on his 2024 Accelerator project, exploring text analysis.

At the Care Inspectorate we collect and manage large volumes of text data which has the potential to offer deep insight to support the delivery of safe, high-quality care. This data is gathered from thousands of inspections taking place across Scotland each year. After each of those inspections, written feedback is provided detailing what is going well and what could be improved, alongside grades corresponding to aspects of the care provided.

As it stands, this large dataset is unwieldy to analyse yet we know holds some of the richest contextual information we have available to us. Our ongoing attempts to dig deeper into analysing this data were often halted by not knowing where to start or shifting priorities in the organisation which meant development work went on the back burner.

This will be a familiar challenge to most public sector analysts – finding both the time and the best technical guidance to implement innovative ideas!

The Accelerator offered a pathway through this dual challenge as it provided access to a skilled mentor to provide technical guidance, while also offering ring-fenced, dedicated time each week to focus the mind.

Having been selected for the 2024 programme, I was lucky enough to be mentored by Tom Wilson, Data Scientist at the Scottish Government who brought a wealth of experience in exploring, analysing, and modelling text information. Tom helped me plan and structure the project from the ground-up. Given we had only 12 days (1 day per week over the 12-week programme), Tom helped me to set realistic goals for what we could achieve.

My hope was to come away with a well-documented reproducible pipeline for exploring text data ahead of modelling. This would mean we could apply methods across our internal datasets and the process could be shared and further developed with the rest of the team.

What the programme taught me was that with the right support, a lot can be achieved in 12 days (or less!).

I already knew that text data is complex and messy, but what I learned was that just because there’s lots of it doesn’t necessarily mean it’s all useful. However, we can absolutely learn from it by using exploratory methods when guided by a strong working hypothesis founded on a deep understanding of the context it sits. Our findings highlighted the importance of data quality and providing feedback to improve data collection processes to make best use of what we collect.

I thoroughly enjoyed the weekly cohort meetings which were an opportunity to discuss projects and arising issues, or exciting findings with other mentees and mentors. The sessions also offered learning about data-related topics such as Git, Agile methodology, ethics, and coding standards.

Now as I look back, I can see we now have a way to explore the vast quantities of text data we hold in a systematic way which was previously a roadblock in using our data to reveal broader trends and patterns. Having presented the findings of the project to colleagues, we’re looking to explore other opportunities for the reproducible pipeline we’ve created and based on the lessons learned during the programme. We hope to continue to build on this work with the next steps looking beyond data exploration at the modelling process to further unlock insights from the text information we collect.

This was just one of many examples where text data could provide insightful for common themes in care across Scotland if aggregated in a systematic way, and this proof-of-concept case was used for this project. If you have a bright idea of how you can use your data in a better way, I encourage you to apply for the Data Science and Innovation Accelerator to bring it to life!

If you’ve been inspired by Sam’s story, please visit: Data Science and Innovation Accelerator – Scottish Digital Academy and get your applications in by 28 April 2025.

If you are interested in supporting people like Sam to progress with their innovative ideas, we’d love you to consider volunteering as an Accelerator mentor. You’ll find all the information you need at Data Science and Innovation Accelerator – Mentor – Scottish Digital Academy.

Tags: Care Inspectorate, Case study, Data Science and Innovation Accelerator, Digital Scotland

Comments