Bovine TB is a complex and costly disease in cattle. Bringing Defra’s data together and providing better access and tools will really help us to tackle it.
Why it matters
Bovine TB is a disease that affects cattle that has been gradually increasing in prevalence across England since the 1980s. It’s hard to spot and the various pathways through which it can be transmitted are complex and hard to separate. To tackle it, Defra runs an extensive programme of testing and this, combined with the cost of removing infected animals and subsequent restrictions imposed on infected farms, costs the taxpayer around £100 million per annum and the industry itself potentially the same again.
What’s the problem?
Defra collects a wealth of data which relates either directly or indirectly to the disease. This includes results of tests, locations of farms, types of practice on the farms, locations and movements of individual animals etc. However, the existing arrangements mean that for internal users actually accessing this data is very constrained and this results in users caching copies when they do get access, altering these copies to improve quality but this results in divergent versions.
The cloud now provides a potential means by which Defra could bring a range of data sources together, regularly refresh them and use these as the primary source for staff. Modern Business Intelligence or 'BI' tools also provide a much more flexible way to interrogate the data.
So the challenge was: could we bring disparate data sources together in the cloud, deploying a BI tool to allow staff to gain new insights into the potential causes of new outbreaks and what can be done to minimise further spread?
Main lessons learned
Overall, the work clearly showed that it was possible and, even on the subset of data and incomplete tooling that there were clear benefits. The vet we worked with was spotting potential issues (things that had been missed) with the existing data. The existing reporting mechanisms are just too onerous to really maximise the application of what we know.
We were working directly with a vet for much of the time and this helped to build a very good understanding of the business need from their perspective. While that still needs more validation with other vets and also greater exploration of the needs of other user groups – it is a very powerful starting point.
The work also helped to drive out some of the potential technical problems. In particular Defra is only just starting to implement a new technical framework within the broader Digital Transformation programme. The organisation's cloud strategy is at an early stage, and this meant it was not really possible to access the tools from within the existing network.
There are still challenges around accessing the data. The sources we were using came from separate organisations and, because they included elements that, with a bit of additional digging, could be used to identify individual farms, the organisations rightly had concerns about their sharing, even internally.
There were also obvious problems around some aspects of the quality of the data. Again this reflects constituent organisations managing data for their own needs rather than looking at what the whole of Defra might find useful.
Overall, and most importantly, the work showed the benefits of making better use of data and the strong desire of the business to push the work forward.
It doesn't stop there
While the focus of the work is on bringing data sources together to tackle Bovine TB, improving access to the same data sources will have a wealth of other benefits. Knowing the densities of cattle and how they are changing is also important for air pollution (cattle are a major source of ammonia and methane) and water quality. The data would also underpin planning broader farm inspections including providing operational staff with the right information during a farm visit.
What happens next?
This work is now moving into an 'alpha' build. During this phase we will:
- implement a more strategic solution to the cloud platform
- completing the business analysis including looking at other areas of the business that use the data such as epidemiology and data science/modelling
- agreeing a robust access policy for the data, including what can be shared externally
- establish mechanism to replicate data from source systems into a cloud environment
- implement a production version of modern reporting tool
The initial focus will be on the generation of reports for each farm, summarising disease history and animal movements for each property to support farmers and external vets in making better decisions at the farm level. The mechanisms we implement for this will allow us to make significant leaps in other areas
This is an excellent opportunity for APHA to move into a smarter and more up-to-date way of treating its data as a key asset, and establishing a single source of information that can be used quickly and flexibly. I’m excited by the possibilities this piece of work could bring to bTB control and prevention, and also how we can use and apply the learning and principles more widely in other areas of APHA.
— Marc Cartwright, Senior Change Manager, Animal and Plant Health Agency (APHA)
Great to see the farm TB data pack being delivered and the prospect for even greater transformation in our use of data to understand bovine TB risks.
— Nigel Gibbens, Chief Veterinary Officer for the UK
