Ten years ago, a community of analysts at the Animal and Plant Health Agency (APHA) started developing an IT system called the Scientific Computing Environment (SCE). Initially a ‘grey IT’ solution, it slowly grew into a complete Defra group system, and has just completed migration into the cloud. Part of what makes the SCE unique is that users can select, install, configure and support their own analytical tools.
The SCE vision is for powerful and agile analytical IT with minimal governance barriers; it trades a degree of support for flexibility and value for money, but users can still build and support their own operational-grade systems. There's plenty of advice and community support available, but apart from ensuring core systems like operating system and network function, the buck stops with the user. Put differently, it's like baking your own cake in a well-stocked kitchen, rather than being limited to what’s on the supermarket shelf.
Delivery hurdles
One of the delivery hurdles was changing user community expectations that IT systems should relieve analysts of systems administration. It’s true there is mileage here, especially with modern data science platforms where many bells and whistles come baked-in.
But any system has limitations - though sometimes you must discover them yourself, the hard way – and just as some lab scientists must develop their novel approaches, some analysts need the IT engineering skills to build bespoke solutions and push boundaries. The SCE team found that communicating service limitations upfront helped frame its unique strengths and drive culture change.
Another challenge was establishing a collaborative relationship with the many corporate IT teams, where the SCE was sometimes perceived as an inappropriate departure from the corporate norm. The delivery team built strong relationships with suppliers such as Strategic Blue, internal security teams and cloud engineers, ultimately demonstrating that the corporate and SCE service models are complementary, yielding a hybrid which is greater than the sum of its parts.
Benefits beyond the expected
The SCE service model was battle-tested in the first 12 months of the COVID-19 outbreak, supporting many urgent unanticipated tasks. Because the SCE team is small and agile, they could design, problem solve, and deploy at speed. We’ve seen benefits including:
- enabling data exchange between laboratory systems undertaking COVID-19 sample testing, and external NHS partners
- developing and hosting COVID-19 status dashboards, for internal audiences and secure sharing with external partners
- enabling rapid provision of resource to support academic COVID-19 modelling
- automating the transfer of whole genome sequence data from on-site laboratories to the cloud for analysis and storage
- universal staff access to more powerful and flexible servers, helping to train and enable an analytical workforce in cases where agility is more critical than support package
- running machine learning workload on clusters of thousands of servers
Reflecting on the journey a key lesson has been this - when evaluating a system, its limitations can provide valuable pointers towards design decisions and strengths you might not have noticed.
Find out more about how APHA has adapted to support the national response to testing human COVID-19 cases.
