{"id":1353,"date":"2022-02-16T12:17:24","date_gmt":"2022-02-16T12:17:24","guid":{"rendered":"https:\/\/blogs.gov.scot\/statistics\/?p=1353"},"modified":"2022-02-21T21:40:38","modified_gmt":"2022-02-21T21:40:38","slug":"improving-the-speed-and-accuracy-of-agricultural-statistics","status":"publish","type":"post","link":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/","title":{"rendered":"Improving the Speed and Accuracy of Agricultural Statistics"},"content":{"rendered":"

This month, the team are looking at improving the speed and accuracy of our agricultural statistics. They focussed on the computer code and the time taken from receiving data to producing statistics. And, we are learning from gaming and software development. In simple terms, it is a way of coding that removes errors. It also makes it easier for future developments to take place.<\/p>\n

Improving Accuracy and Replacing ‘Copy and Paste’<\/h3>\n

Up until now, analysts have had to edit the code in June Agricultural Census<\/a> and upload the data into our analytical software. Think \u2018copy and paste\u2019 on an industrial scale! This can, and does, lead to errors. It takes a huge amount of time for them to then find and correct them.<\/p>\n

The solution is Reproducible Analytical Pipelines or RAPs, a specific method for writing statistical analysis code. This blog on RAPs and data in government<\/a> explains the method in greater detail.<\/p>\n

We are adopting RAP principles to reduce, eliminate or identify errors more easily. RAP also makes the code more easily understood. Done properly, this will save time on error checking and data processing. And future improvements will be easier to make.<\/p>\n

Another crucial benefit is that they can test the code with live data. Because of the way that missing data is handled, re-running code can yield results that differ each time it is run. Instead, we will feed data into a well-tested and notated algorithm. An algorithm is just a fancy name for many different computing processes run one after the other or at the same time.<\/p>\n

In theory these changes should speed up the time taken to produce analysis. This will give our teams more time to go deeper into the data and learn more about what it is telling us.<\/p>\n

More Open Code<\/h3>\n

And the team have taken it a step further. Again, similar to gaming and software development the team are using a Git repository. This is a method of version control, where teams can code in a virtual space, review and make changes without overwriting any work. This system then \u2018pulls\u2019 the work into a final version. <\/p>\n

Our aim is to \u201ccode in the open\u201d similar to the Scottish Crop Map<\/a>. We will be storing our code on GitHub<\/a> again. Meaning, any researcher or student will be able to go in and examine the raw code. Anyone can see how the data from June Agricultural Census is turned into the final product of statistics. You will be able to examine the algorithms and even make suggestions to improve it, so we have effective and efficient code.<\/p>\n

So far the team have practiced on a series of \u2018mini-projects\u2019. First we used our pig data then moved on to cereals, improving the speed and accuracy of statistics. Results look promising and the speed at which the data can be taken to produce visual tables, charts and analysis is already a improving our current system.<\/p>\n

If this is our first jump into expanding our capability to replace the ageing systems and surveys with modern techniques, I am starting to get really excited!<\/p>\n

Get Involved<\/h3>\n

Are you a researcher in agriculture or land use? Why not get involved? Contact the team<\/a> and join us in one of our \u2018Show and Tell\u2019 sessions.<\/p>\n","protected":false},"excerpt":{"rendered":"

This month, the team are looking at improving the speed and accuracy of our agricultural statistics. They focussed on the computer code and the time taken from receiving data to producing statistics. And, we are learning from gaming and software development. In simple terms, it is a way of coding that removes errors. It also…<\/p>\n","protected":false},"author":604,"featured_media":1356,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15,129,1,127],"tags":[18,149,10,19,5,43],"class_list":["post-1353","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-farming-and-rural","category-land-use","category-uncategorized","category-working-in-statistics","tag-agriculture","tag-analytical-pipelines","tag-data","tag-farming","tag-statistics","tag-survey"],"yoast_head":"\nImproving the Speed and Accuracy of Agricultural Statistics - Statistics<\/title>\n<meta name=\"description\" content=\"How using techniques from software development are improving the speed and accuracy of agricultural statistics, using RAPS\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Improving the Speed and Accuracy of Agricultural Statistics - Statistics\" \/>\n<meta property=\"og:description\" content=\"How using techniques from software development are improving the speed and accuracy of agricultural statistics, using RAPS\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/\" \/>\n<meta property=\"og:site_name\" content=\"Statistics\" \/>\n<meta property=\"article:published_time\" content=\"2022-02-16T12:17:24+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-02-21T21:40:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"230\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Ally McAlpine\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@StattoAlly\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ally McAlpine\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/\"},\"author\":{\"name\":\"Ally McAlpine\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/0eb925405a8a1ea9a8e58cb7da6ae704\"},\"headline\":\"Improving the Speed and Accuracy of Agricultural Statistics\",\"datePublished\":\"2022-02-16T12:17:24+00:00\",\"dateModified\":\"2022-02-21T21:40:38+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/\"},\"wordCount\":554,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg\",\"keywords\":[\"agriculture\",\"analytical pipelines\",\"data\",\"farming\",\"statistics\",\"Survey\"],\"articleSection\":[\"Farming and rural\",\"Land use\",\"Uncategorized\",\"Working in statistics\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/\",\"url\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/\",\"name\":\"Improving the Speed and Accuracy of Agricultural Statistics - Statistics\",\"isPartOf\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg\",\"datePublished\":\"2022-02-16T12:17:24+00:00\",\"dateModified\":\"2022-02-21T21:40:38+00:00\",\"author\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/0eb925405a8a1ea9a8e58cb7da6ae704\"},\"description\":\"How using techniques from software development are improving the speed and accuracy of agricultural statistics, using RAPS\",\"breadcrumb\":{\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage\",\"url\":\"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg\",\"contentUrl\":\"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg\",\"width\":800,\"height\":230,\"caption\":\"Sunrise over a field of vegetable crops\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/blogs.gov.scot\/statistics\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Improving the Speed and Accuracy of Agricultural Statistics\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/#website\",\"url\":\"https:\/\/blogs.gov.scot\/statistics\/\",\"name\":\"Statistics\",\"description\":\"Scottish Government Blog\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blogs.gov.scot\/statistics\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/0eb925405a8a1ea9a8e58cb7da6ae704\",\"name\":\"Ally McAlpine\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/blogs.gov.scot\/statistics\/wp-content\/wphb-cache\/gravatar\/a1f\/a1f07b9e359b5406fad8233aaa81d6f2x96.jpg\",\"contentUrl\":\"https:\/\/blogs.gov.scot\/statistics\/wp-content\/wphb-cache\/gravatar\/a1f\/a1f07b9e359b5406fad8233aaa81d6f2x96.jpg\",\"caption\":\"Ally McAlpine\"},\"description\":\"Alastair is the Chief Statistician in the Scottish Government. He is responsible for the professional standards of official statistics produced in Scotland, their trustworthiness, value and quality. He is also co-Director at ADR Scotland which is a partnership combining specialists in the Scottish Government\u2019s Data for Research Unit with the expertise of academic researchers at the Scottish Centre for Administrative Data Research.\",\"sameAs\":[\"https:\/\/x.com\/StattoAlly\"],\"url\":\"https:\/\/blogs.gov.scot\/statistics\/author\/alastairmcalpine\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Improving the Speed and Accuracy of Agricultural Statistics - Statistics","description":"How using techniques from software development are improving the speed and accuracy of agricultural statistics, using RAPS","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/","og_locale":"en_GB","og_type":"article","og_title":"Improving the Speed and Accuracy of Agricultural Statistics - Statistics","og_description":"How using techniques from software development are improving the speed and accuracy of agricultural statistics, using RAPS","og_url":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/","og_site_name":"Statistics","article_published_time":"2022-02-16T12:17:24+00:00","article_modified_time":"2022-02-21T21:40:38+00:00","og_image":[{"width":800,"height":230,"url":"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg","type":"image\/jpeg"}],"author":"Ally McAlpine","twitter_card":"summary_large_image","twitter_creator":"@StattoAlly","twitter_misc":{"Written by":"Ally McAlpine","Estimated reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#article","isPartOf":{"@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/"},"author":{"name":"Ally McAlpine","@id":"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/0eb925405a8a1ea9a8e58cb7da6ae704"},"headline":"Improving the Speed and Accuracy of Agricultural Statistics","datePublished":"2022-02-16T12:17:24+00:00","dateModified":"2022-02-21T21:40:38+00:00","mainEntityOfPage":{"@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/"},"wordCount":554,"commentCount":0,"image":{"@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage"},"thumbnailUrl":"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg","keywords":["agriculture","analytical pipelines","data","farming","statistics","Survey"],"articleSection":["Farming and rural","Land use","Uncategorized","Working in statistics"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/","url":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/","name":"Improving the Speed and Accuracy of Agricultural Statistics - Statistics","isPartOf":{"@id":"https:\/\/blogs.gov.scot\/statistics\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage"},"image":{"@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage"},"thumbnailUrl":"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg","datePublished":"2022-02-16T12:17:24+00:00","dateModified":"2022-02-21T21:40:38+00:00","author":{"@id":"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/0eb925405a8a1ea9a8e58cb7da6ae704"},"description":"How using techniques from software development are improving the speed and accuracy of agricultural statistics, using RAPS","breadcrumb":{"@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#primaryimage","url":"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg","contentUrl":"https:\/\/blogs.gov.scot\/statistics\/wp-content\/uploads\/sites\/24\/2022\/02\/47741667031_81834bce06_c-e1645012632718.jpg","width":800,"height":230,"caption":"Sunrise over a field of vegetable crops"},{"@type":"BreadcrumbList","@id":"https:\/\/blogs.gov.scot\/statistics\/2022\/02\/16\/improving-the-speed-and-accuracy-of-agricultural-statistics\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blogs.gov.scot\/statistics\/"},{"@type":"ListItem","position":2,"name":"Improving the Speed and Accuracy of Agricultural Statistics"}]},{"@type":"WebSite","@id":"https:\/\/blogs.gov.scot\/statistics\/#website","url":"https:\/\/blogs.gov.scot\/statistics\/","name":"Statistics","description":"Scottish Government Blog","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blogs.gov.scot\/statistics\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/0eb925405a8a1ea9a8e58cb7da6ae704","name":"Ally McAlpine","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/blogs.gov.scot\/statistics\/#\/schema\/person\/image\/","url":"https:\/\/blogs.gov.scot\/statistics\/wp-content\/wphb-cache\/gravatar\/a1f\/a1f07b9e359b5406fad8233aaa81d6f2x96.jpg","contentUrl":"https:\/\/blogs.gov.scot\/statistics\/wp-content\/wphb-cache\/gravatar\/a1f\/a1f07b9e359b5406fad8233aaa81d6f2x96.jpg","caption":"Ally McAlpine"},"description":"Alastair is the Chief Statistician in the Scottish Government. He is responsible for the professional standards of official statistics produced in Scotland, their trustworthiness, value and quality. He is also co-Director at ADR Scotland which is a partnership combining specialists in the Scottish Government\u2019s Data for Research Unit with the expertise of academic researchers at the Scottish Centre for Administrative Data Research.","sameAs":["https:\/\/x.com\/StattoAlly"],"url":"https:\/\/blogs.gov.scot\/statistics\/author\/alastairmcalpine\/"}]}},"_links":{"self":[{"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/posts\/1353","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/users\/604"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/comments?post=1353"}],"version-history":[{"count":0,"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/posts\/1353\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/media\/1356"}],"wp:attachment":[{"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/media?parent=1353"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/categories?post=1353"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gov.scot\/statistics\/wp-json\/wp\/v2\/tags?post=1353"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}