{"id":5460,"date":"2025-04-23T09:33:07","date_gmt":"2025-04-23T09:33:07","guid":{"rendered":"https:\/\/blogs.gov.scot\/digital\/?p=5460"},"modified":"2025-04-23T09:33:07","modified_gmt":"2025-04-23T09:33:07","slug":"tackling-child-poverty-with-machine-learning","status":"publish","type":"post","link":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/","title":{"rendered":"Tackling child poverty with machine learning"},"content":{"rendered":"\r\n

Blog post by Spencer Thompson, Economic Advisor, Scottish Government<\/strong><\/p>\r\n

The Data Science and Innovation Accelerator is a programme for public sector organisations in Scotland that supports the innovative use of data. In his blog, Spencer Thompson discusses his 2024 Accelerator project on child poverty. \"\"<\/a><\/p>\r\n

\u201cOver the past 6 years I\u2019ve been working towards the Scottish Government\u2019s mission of tackling child poverty. During this time I\u2019ve been haunted by the most basic of questions – who are the children that live in poverty in our country? The question may be basic, but it\u2019s by no means simple to answer.<\/p>\r\n

According to the latest child poverty statistics<\/a>, nearly one in four children in Scotland are living in relative poverty. Of those, around nine out of ten fall into at least one of the \u2018priority groups\u2019 as defined by age, ethnicity, disability, and family size and structure. Many other characteristics are also relevant, depending on the context \u2013 and all of these characteristics intersect.<\/p>\r\n

But how can we make sense of the resulting complexity? Standard cross-tabulations only gets us so far. The number of possible combinations will increase exponentially with the number of characteristics, so it doesn\u2019t take much before we have exceeded what the human brain can grasp at an intuitive level. The priority group characteristics alone contain 64 possible combinations \u2013 and that\u2019s if we treat them as simple \u2018yes\/no\u2019 binaries.<\/p>\r\n

The Accelerator gave me the opportunity to seek answers to my questions. Having completed the ONS Data Science Graduate Programme the year before, it was the perfect follow-up, allowing me to put my learning to work in the service of one of the Scottish Government\u2019s chief priorities \u2013 and learning a lot more in the process.<\/p>\r\n

In particular, I set out to discover whether machine learning techniques can help us identify distinct \u2018clusters\u2019 of households in poverty. The idea is that each household with children in poverty is categorised into one, but only one, of a number of clusters. If the method has worked well, households within each cluster should be similar to each other, while households in different clusters should be dissimilar to each other. By characterising these clusters, we can provide a simplified but rigorous answer to my original question: who are the children that are living in poverty?<\/p>\r\n

The idea is straightforward enough, but implementing it turned out to be more complicated than I\u2019d envisaged. The guidance of my mentor, Chenglei Hu, was invaluable in navigating the various challenges that I encountered. We made a great team, with my knowledge of the data and the policy area combining with his knowledge of cutting-edge methods and approaches.<\/p>\r\n

Besides picking up some valuable new skills, a key learning point for me was to be open-minded. I was initially intent on designing a fully unsupervised machine-learning task, where I \u2018let the data speak\u2019 with minimal instruction or intervention. But this led me down a number of dead ends; and it was only when I heeded Chenglei\u2019s advice to introduce an element of supervision that I was able to find a way forward.<\/p>\r\n

Another lesson for me was not to lose sight of the \u2018why\u2019 in the midst of the \u2018what\u2019 and the \u2018how\u2019. We ended up layering on several different methods, which allowed us to solve technical problems and produce a robust set of results. But by the same token, interpreting and communicating these results \u2013 and ultimately using them to inform policy \u2013 requires more care and attention.<\/p>\r\n

And finally, to affirm that old clich\u00e9, I was assured that it\u2019s okay to ask daft questions \u2013 in fact, it\u2019s necessary. To put it somewhat loftily, in the supposed words of Erwin Schr\u00f6dinger, the great physicist of quantum cat fame: \u2018the task is not so much to see what no one has yet seen; but to think what nobody has yet thought, about that which everybody sees.\u2019 If we are to achieve our ambitious goal of tackling child poverty, we will need all the daft questions we can get.\u201d<\/p>\r\n

If you\u2019ve been inspired by Spencer\u2019s quest for understanding and want to uncover the stories in your data, visit: Data Science and Innovation Accelerator – Scottish Digital Academy<\/a> and apply by 28 April for our 2025 programme.<\/p>\r\n

If you are interested in supporting people to progress the answers to important societal questions, we\u2019d love you to consider volunteering as an Accelerator mentor. You\u2019ll find all the information you need at Data Science and Innovation Accelerator – Mentor – Scottish Digital Academy<\/a>.<\/p>\r\n","protected":false},"excerpt":{"rendered":"

Blog post by Spencer Thompson, Economic Advisor, Scottish Government The Data Science and Innovation Accelerator is a programme for public sector organisations in Scotland that supports the innovative use of data. In his blog, Spencer Thompson discusses his 2024 Accelerator project on child poverty. \u201cOver the past 6 years I\u2019ve been working towards the Scottish…<\/p>\n","protected":false},"author":317,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[480,6],"tags":[485,481,95],"class_list":["post-5460","post","type-post","status-publish","format-standard","hentry","category-data-science-and-innovation-accelerator","category-digital-scotland","tag-child-poverty-statistics","tag-data-science-and-innovation-accelerator","tag-scottish-government"],"yoast_head":"\nTackling child poverty with machine learning - Digital<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Tackling child poverty with machine learning - Digital\" \/>\n<meta property=\"og:description\" content=\"Blog post by Spencer Thompson, Economic Advisor, Scottish Government The Data Science and Innovation Accelerator is a programme for public sector organisations in Scotland that supports the innovative use of data. In his blog, Spencer Thompson discusses his 2024 Accelerator project on child poverty. \u201cOver the past 6 years I\u2019ve been working towards the Scottish...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/\" \/>\n<meta property=\"og:site_name\" content=\"Digital\" \/>\n<meta property=\"article:published_time\" content=\"2025-04-23T09:33:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson-300x300.jpg\" \/>\n<meta name=\"author\" content=\"Stewart Hamilton\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Stewart Hamilton\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/\"},\"author\":{\"name\":\"Stewart Hamilton\",\"@id\":\"https:\/\/blogs.gov.scot\/digital\/#\/schema\/person\/fd47935c780321ad6c4ecbb2f10da552\"},\"headline\":\"Tackling child poverty with machine learning\",\"datePublished\":\"2025-04-23T09:33:07+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/\"},\"wordCount\":762,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson-300x300.jpg\",\"keywords\":[\"child poverty statistics\",\"Data Science and Innovation Accelerator\",\"scottish government\"],\"articleSection\":[\"Data Science and Innovation Accelerator\",\"Digital Scotland\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/\",\"url\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/\",\"name\":\"Tackling child poverty with machine learning - Digital\",\"isPartOf\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson-300x300.jpg\",\"datePublished\":\"2025-04-23T09:33:07+00:00\",\"author\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/#\/schema\/person\/fd47935c780321ad6c4ecbb2f10da552\"},\"breadcrumb\":{\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage\",\"url\":\"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson.jpg\",\"contentUrl\":\"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson.jpg\",\"width\":1024,\"height\":1024},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blogs.gov.scot\/digital\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Tackling child poverty with machine learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blogs.gov.scot\/digital\/#website\",\"url\":\"https:\/\/blogs.gov.scot\/digital\/\",\"name\":\"Digital\",\"description\":\"Updates from the Scottish Government's Digital Directorate\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blogs.gov.scot\/digital\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blogs.gov.scot\/digital\/#\/schema\/person\/fd47935c780321ad6c4ecbb2f10da552\",\"name\":\"Stewart Hamilton\",\"description\":\"Communications and Engagement Officer\",\"url\":\"https:\/\/blogs.gov.scot\/digital\/author\/stewarthamilton\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Tackling child poverty with machine learning - Digital","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/","og_locale":"en_GB","og_type":"article","og_title":"Tackling child poverty with machine learning - Digital","og_description":"Blog post by Spencer Thompson, Economic Advisor, Scottish Government The Data Science and Innovation Accelerator is a programme for public sector organisations in Scotland that supports the innovative use of data. In his blog, Spencer Thompson discusses his 2024 Accelerator project on child poverty. \u201cOver the past 6 years I\u2019ve been working towards the Scottish...","og_url":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/","og_site_name":"Digital","article_published_time":"2025-04-23T09:33:07+00:00","og_image":[{"url":"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson-300x300.jpg","type":"","width":"","height":""}],"author":"Stewart Hamilton","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Stewart Hamilton","Estimated reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#article","isPartOf":{"@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/"},"author":{"name":"Stewart Hamilton","@id":"https:\/\/blogs.gov.scot\/digital\/#\/schema\/person\/fd47935c780321ad6c4ecbb2f10da552"},"headline":"Tackling child poverty with machine learning","datePublished":"2025-04-23T09:33:07+00:00","mainEntityOfPage":{"@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/"},"wordCount":762,"commentCount":0,"image":{"@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage"},"thumbnailUrl":"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson-300x300.jpg","keywords":["child poverty statistics","Data Science and Innovation Accelerator","scottish government"],"articleSection":["Data Science and Innovation Accelerator","Digital Scotland"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/","url":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/","name":"Tackling child poverty with machine learning - Digital","isPartOf":{"@id":"https:\/\/blogs.gov.scot\/digital\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage"},"image":{"@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage"},"thumbnailUrl":"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson-300x300.jpg","datePublished":"2025-04-23T09:33:07+00:00","author":{"@id":"https:\/\/blogs.gov.scot\/digital\/#\/schema\/person\/fd47935c780321ad6c4ecbb2f10da552"},"breadcrumb":{"@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#primaryimage","url":"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson.jpg","contentUrl":"https:\/\/blogs.gov.scot\/digital\/wp-content\/uploads\/sites\/5\/2025\/04\/Data-Science-Accelerator-2024-Spencer-Thomson.jpg","width":1024,"height":1024},{"@type":"BreadcrumbList","@id":"https:\/\/blogs.gov.scot\/digital\/2025\/04\/23\/tackling-child-poverty-with-machine-learning\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blogs.gov.scot\/digital\/"},{"@type":"ListItem","position":2,"name":"Tackling child poverty with machine learning"}]},{"@type":"WebSite","@id":"https:\/\/blogs.gov.scot\/digital\/#website","url":"https:\/\/blogs.gov.scot\/digital\/","name":"Digital","description":"Updates from the Scottish Government's Digital Directorate","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blogs.gov.scot\/digital\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/blogs.gov.scot\/digital\/#\/schema\/person\/fd47935c780321ad6c4ecbb2f10da552","name":"Stewart Hamilton","description":"Communications and Engagement Officer","url":"https:\/\/blogs.gov.scot\/digital\/author\/stewarthamilton\/"}]}},"_links":{"self":[{"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/posts\/5460","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/users\/317"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/comments?post=5460"}],"version-history":[{"count":0,"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/posts\/5460\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/media?parent=5460"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/categories?post=5460"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gov.scot\/digital\/wp-json\/wp\/v2\/tags?post=5460"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}