Defra group has the potential to simplify its access to satellite data and transform how we monitor and manage the UK’s landscape.
A big proportion of our work in the Defra group is focused around how the UK uses its environment – from fields, to forests, to seas, and beyond. As part of this work we monitor how our environment changes over time, allowing us to make evidence-based decisions in the future.
As a department, we spend a large proportion of our budget on our monitoring activities, which can often involve travelling to places to physically inspect them. We're always exploring ways in which we can do this through better use of data and technology. In an ideal world we would target our inspections where they’re most needed, and do that based on an assessment of the best data we have available.
The UK already has access to data from the European Space Agency’s Copernicus Programme. The programme sources ‘Earth observation’ data from a suite of satellites, each with different orbits around the Earth that between them produce an overall picture of the health of our planet. Sentinel 1 and Sentinel 2 satellites pass over the UK every two to three days, giving Defra an up-to-date picture of the environment and any changes that occur in it. In Defra, we already use some of this data, for example to create crop maps to help target inspections by detecting bare earth as sources of potential water quality problems, or to monitor water levels in flood management.
What’s the challenge?
Processing Earth observation data takes time and is costly. Not only is the data from the satellites huge in terms of their file size (each Sentinel-1 produces 1.8TB of data per day; each Sentinel-2 1.6TB; that’s roughly the same as 300,000 smartphone photos all in one file), but it also takes time to get the raw data from the satellites into a format that’s usable by users across the Defra group. Often that means cleaning up and filtering out things like atmospheric distortion or cloud layers. We currently do this on an individual case-by-case basis, which is both costly in terms of time and effort.
If we could do this once by establishing a standard way to process the cleaning up and filtering out part, we could apply it to our existing uses of the data as well as new and innovative ways of monitoring in Defra. We could keep our running costs down, as well as removing barriers for new users to apply Earth observation data-led analysis to their areas of work.
What did we set out to do?
The Defra Data Transformation team worked on a prototype with Defra’s Earth Observation Centre of Excellence to see whether we could fulfil that challenge. We did this by identifying common processes most users of the data would need before analysing the data and carrying them out beforehand, enabling Defra staff to do their work more effectively and efficiently.
What did we do?
- We produced a common tool, based on user needs, that could be reused.
Some 80% of uses of Earth observation data across Defra require a surface reflectance product – essentially, a good quality image of the surface of the land. Most uses of the data rely on a good image of the Earth's surface but the data get distorted as the signal goes through the atmosphere. We developed a common tool that used a dark object in an image as a point of reference, from which the effects of atmospheric distortion can be identified and removed for the whole image.
- We automated initial processing steps, freeing up time to focus on developing more advanced analysis and uses of satellite data.
This means approximately 50–70% of specialist time required for the initial process, from acquisition and downloading, to cleaning, can be used to make sure we’re getting the most from this data.
- We showed that carrying out the initial processing in one centralised way increases efficiency, both in terms of cost and people’s time.
Combined with open source technologies (particularly Geoserver), this meant that a single data source could be used across the Defra group, as well as being accessible through standard software tools.
What did we learn?
By working with Defra’s Earth Observation Centre of Excellence, we were able to
- test the power of working collaboratively to solve a common problem
- build a good prototype that allowed a common source of the data to be directly accessed and used across the Defra group
- show the potential of what a data-driven solution could be for the organisation, including the use of a cloud-based service
- tell the story of what we’ve discovered in the prototyping, allowing us to gauge the organisation’s demand to move on to a productionised version.
What happens now?
While Defra already uses satellite Earth observation data, we’re just scratching the surface of the possibilities and its benefits. The coming years will see the launch of more satellites with sensors that are more relevant to detecting changes in the seas and atmosphere. These will be useful in expanding on the policy and operation areas where Earth observation data can be used.
In the immediate term, we’re moving this work into an alpha build. During this phase we’ll:
- implement a more strategic cloud solution, allowing users across Defra to access the data directly
- build fully automated routines to download and clean the raw satellite data
- provide access to the data to general staff across the Defra group and tools to allow them to visualise it, and
- enable Defra data scientists to analyse the data and produce products that help transform policy delivery.
There are a variety of potential uses of the satellite data which have been proven but not yet explored; processing once will allow these to be quickly used everywhere.
