Python for Data

From Spreadsheet Chaos to Genomics: How a Glasgow Gastroenterologist Built an AI-Powered Data Platform

In 2020, Dr. Shaun Chuah, a gastroenterologist and clinical researcher based at the University of Glasgow, faced a monumental data logistics challenge. Embarking on a wide-ranging multi-city research study investigating inflammatory bowel disease (IBD)—specifically conditions such as Crohn’s disease and ulcerative colitis—Chuah calculated the project’s operational scale. Over a two-year period, the research initiative would draw from dozens of hospitals across three major urban centers, ultimately yielding approximately 30,000 biological samples.

When Chuah inquired within his network about how contemporary clinical researchers tracked such vast volumes of physical specimens and associated data, the overwhelming consensus pointed to a rudimentary tool: Microsoft Excel. Recognizing that manual spreadsheets would rapidly buckle under the weight of thousands of clinical records, paired with his own rudimentary programming background limited to hand-coded HTML written in Notepad during his high school years, Chuah decided to take an unconventional path. He opened a Django tutorial, began reading software development documentation, and started building a custom web application. Six years later, that fledgling web project has matured into Foundry120, a robust clinical data management platform holding 10 terabytes of complex clinical and genomic data, backed by a specialized agentic artificial intelligence layer known as Helix.

The Genesis of Foundry120 and the Modern Clinical Data Bottleneck

The hurdles facing medical researchers often extend far beyond the laboratory bench into the realm of administrative and data infrastructure. In the early phases of the IBD study, Chuah’s team needed a system capable of registering participant profiles, tracking physical samples via QR codes, and tracing the lifecycle of biological materials as they moved between hospital sites and research laboratories. Off-the-shelf enterprise laboratory information management systems (LIMS) proved financially prohibitive, notoriously slow to deploy, and overly rigid, typically engineered for routine hospital operations rather than agile, multi-site academic studies.

Faced with these constraints, Chuah turned to the Python ecosystem and the Django web framework. Drawn to Django’s "batteries-included" philosophy, built-in security features, and robust database migration management, he successfully launched the initial application to support the multi-center study. What began as a simple sample-tracking registry quickly expanded its scope. As biological samples were processed through various assays, they generated massive influxes of digital data—ranging from microbiome sequencing and genotyping files to high-resolution endoscopy videos, digital pathology slides, and cell-free DNA fragments. Individual genomic sequencing files alone routinely reached five to ten gigabytes per participant, rapidly transforming a local tracking tool into a massive data warehousing challenge.

Scaling Infrastructure and Migrating to Cloud Architecture

As the volume of clinical and genomic data scaled toward tens of terabytes, local university file shares and capped institutional servers became untenable. Research teams frequently encountered storage quotas, forcing cumbersome requests to institutional IT departments and creating fragmented data silos scattered across individual laptops and shared network drives.

To resolve these bottlenecks, the Foundry120 architecture underwent a major evolution, migrating its core storage infrastructure to Microsoft Azure Blob Storage. This shift replaced hard capacity limits with a scalable cloud-based model hosted primarily within Azure’s UK South and Sweden data centers, satisfying stringent regional data governance requirements such as GDPR.

Furthermore, the integration of continuous integration and continuous deployment (CI/CD) pipelines via GitHub Actions allowed Chuah and his collaborators to deploy updates and bug fixes to production environments within minutes. By centralizing disparate data modalities into a unified cloud repository, the platform laid the groundwork for advanced computational data analysis, setting the stage for the integration of modern artificial intelligence.

The Introduction of Agentic AI: Helix

As data accumulation accelerated, traditional querying methods—such as manual database joins performed by human researchers—created significant operational latency. Finding specific cohorts of participants based on complex cross-variable parameters could take days of manual scripting and data wrangling. To address this, Chuah and his team developed Helix, an agentic AI system embedded within the Foundry120 framework.

Unlike standard conversational chatbots that function primarily as information retrieval tools, Helix operates as a tool-using agent capable of executing multi-step computational tasks. When a researcher submits a natural language query—such as requesting a breakdown of plasma samples categorized by specific disease classifications—Helix assesses available tools, writes and executes Python analysis scripts within secure virtual machine sandboxes, reviews output logs, handles runtime errors, and synthesizes final reports or data visualizations.

This agentic paradigm fundamentally shifts the daily responsibilities of laboratory personnel. Rather than spending weeks writing exploratory data analysis code or manually cross-referencing records, researchers can delegate routine data extraction and preliminary analysis to the AI agent, shifting their focus toward empirical verification and scientific validation.

Security, Governance, and Sandboxing Protocols

Deploying artificial intelligence within clinical research environments introduces critical security and compliance challenges, particularly regarding patient privacy, institutional data governance, and regulatory frameworks such as HIPAA and GDPR. Unauthorized data exposure or unintended model access can compromise institutional compliance and breach research ethics approvals.

To mitigate these risks, Foundry120 implements a rigorous security hierarchy. The platform utilizes the Django backend as a secure intermediary layer or "guardrail" between the AI agent and the underlying raw data. Helix does not interact directly with raw storage or production databases; instead, it operates through strictly defined, deterministic tools authorized by the application framework.

Additionally, computational tasks requiring heavy data processing are executed within isolated, ephemeral virtual machine sandboxes on Azure. Role-based access controls (RBAC) enforced by Django ensure that the AI agent can only access and analyze data files matching the specific permission levels of the authenticated researcher submitting the query. This architectural design preserves data confidentiality while harnessing the computational efficiency of cloud-based AI.

Broader Implications for Translational Science

The development and deployment of Foundry120 highlight a broader technological shift occurring across academic research institutions. As modern biological assays generate exponentially larger data sets—with some advanced sequencing technologies approaching terabyte-scale outputs per sample—traditional data management strategies are rapidly reaching their limits.

While enterprise software solutions often remain cost-prohibitive for academic research groups, the accessibility of modern web frameworks like Django and scalable cloud infrastructure enables clinicians and domain experts to build bespoke, highly functional platforms tailored directly to their operational needs. Moreover, the integration of agentic AI workflows demonstrates a viable path toward streamlining complex data analysis pipelines, potentially shortening the lengthy timelines traditionally associated with translational medicine and therapeutic discovery.

Looking forward, platforms like Foundry120 suggest a future where domain specialists can leverage localized, secure AI agents to navigate massive institutional data repositories without sacrificing regulatory compliance. As these technologies mature, their successful implementation will depend heavily on maintaining rigorous human oversight, robust security sandboxing, and strict adherence to data governance standards across the global research community.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button