IT Flexibility Versus Continuous Compliance for Clinical Trials
June 2022
Summary
Using modifications to modern software strategies, clinical research IT teams can update systems quickly to meet modern research requirements without compromising quality or compliance. Under both current and pending regulatory guidance, system changes need validation and this deters sponsors and CROs from either making the changes needed for optimal study execution or validating the changes due to cost. However, modifying iterative software techniques (i.e., Agile) to quickly deliver change while generating compliance outputs is achievable including validated study tools on 3-week delivery schedules.
Background: Research and Technology Needs Both Outpace Regulatory Requirements
Traditional clinical research advanced through formal milestones across longer schedules with rare changes to study conduct, statistical models, or protocols. Likewise, traditional software and validation processes allowed months for the development, validation, and documentation of clinical systems to fulfill the documentation of regulatory requirements for systems validation. In the last two decades, validation for regulated clinical (i.e., “GxP”) systems needed to demonstrate the application was “fit for intended use” and provided proof of sufficient testing through auditable records. This commonly translated to sponsors and CROs using “traditional validation” strategies (e.g., the “certainty model” of waterfall phases and artifacts) to create validation documents or "binders" for each release that show the comprehensive User Requirement Specification with its requisite reviews and approvals as the requirements describing intended use and then testing to show that all features have been successfully verified against the stated requirements. Traditional validation uses the Validation Plans, Summary Reports, and Installation/Operations/Performance Qualification (IQ/OQ/PQ) phases that many of us are familiar with to test the full suite of system and functional requirements.
Modern Research Requires Faster Change
However, clinical research changes are accelerated with events like the COVID-19 pandemic. More trials have quickly adopted remote monitoring and decentralized clinical trial models (e.g., using monitoring devices and wearables vs traditional site visits) to improve patient enrollment, retention, and safety during trials. Study enrollment strategies shift in pandemic populations or with rapidly mutating viruses. To meet changing study portfolios, clinical providers adopt new technologies mid-study like artificial intelligence and machine learning, blockchain, and virtual reality. These and other examples reflect a fundamental change in clinical research that is not a temporary byproduct of a pandemic but a transformative shift in how our industry envisions research. In this environment, change increases demands on sponsors, CROs, manufacturers and other stakeholders in trials to quickly upgrade IT systems while demonstrating continuous compliance with regulatory authority requirements (e.g., FDA systems validation, 21 CFR Part 11, ICH GxP, etc.)
Modern Software Promises Faster Change
Even outside of pandemic challenges, today’s software industry promises customers bespoke solutions better and faster than older platforms and promise configurability without clarifying compliance. Software provides immediate access to data and dashboards and configurations that can deliver tailored experiences to study coordinators, medical monitors, investigators or the myriad of other clinical roles. System users in modern programs create reports or views without considering potential compliance impacts (e.g., creating a dashboard or report that was used in an IRB meeting). Similarly, for both large programs and especially for smaller pharma companies looking to enter the market, Software-as-a-Service (SaaS) providers that host and configure online applications for clinical providers promise clinical platforms with lower entry cost that enforce standards while offering comprehensive configurability and speeding implementation timelines. SaaS providers offer clinical software without any installation and hosting requirements for customers and commonly claim to reduce compliance burdens as a result though these solutions present other challenges including securing data in shared environments and managing multiple mandatory updates each year.
Regulatory Authorities Require Validation of Change
However, clinical trial sponsors must always meet the obligation to regulatory authorities: to ensure that every system is fully validated to protect the confidentiality, integrity, and availability of trial information in order to recreate the study and that the system is suitable for its intended use, including proper system user training. Every change to the system must be evaluated through this lens and documented to demonstrate continuous compliance. Whether using complex integrated sponsor systems or simplified SaaS tools, sponsors must still demonstrate compliance with each release through validation and documentation even when changes are required quickly or implemented by a SaaS provider.
Problem Statement
Sponsors and other clinical stakeholders have stalled system improvements due to cost and compliance burdens imposed by traditional GxP validation. Due in part to this negative trend, regulatory authorities like the FDA and EMA have pending changes for 2022 and 2023. FDA’s new guidance topic, “Computer Software Assurance for Production and Quality System Software”, may soon be published and seeks to streamline validation based in part on the International Society of Pharmaceutical Engineers (ISPE) GAMP® 5 model and may clarify compliance requirements and reduce documentation burdens currently demanded by traditional validation. A pending 2022 update from the EMA (“Guideline on computerised systems and electronic data in clinical trials”) provides similar updates on modern data governance. However, sponsor responsibilities are largely unchanged; while these new guidance documents may reduce some efforts in traditional validation by using risk-based techniques to prioritize those features impacting patient safety and product quality, they essentially restructure but do not eliminate the sponsor’s obligations.
How can clinical IT manage heavily-regulated clinical systems including clinical trial, data, and safety management platforms to accommodate updates in weeks while still meeting continuous compliance demands from regulatory authorities?
Challenges in Compliance
Both the traditional validation and emerging guidance requirements present challenges to sponsors and CROs and the clinical IT providers serving them. The most common challenges that act as barriers to effective clinical informatics are:
- Documentation Burdens
- Awareness and the “Buzz”
- Change Controls and Responsibilities
- Cost Constraints
Documentation Burdens
Traditional validation results in sponsors and clinical IT teams spending around 80% of their time producing documentation and only 20% of their time doing actual testing of the software. A major clinical system may require 400+ hours just to complete validation documentation for each release. Most major commercial software publishers sell “validation kits” including templates for sponsors or CROs to reduce the creation cost (but not executions and approvals). SaaS providers provide evidence of proper installation and maintenance, but the functional configuration and use of the tool must still be validated by the customer. In brief, traditional validation often follows common IQ/OQ/PQ and GAMP® 5 strategies which still impose extensive documentation on risk, requirements, traceability, testing and other artifacts for each change.
Awareness and the “Buzz”
As industry is aware of the compliance burdens summarized above especially considering new clinical realities and technology innovations, many CROs and clinical IT providers brand themselves using modern buzzwords like “agile” and “transformative” to attract investment without really understanding or executing on the change needed to deliver research faster in our new model. Similarly, SaaS providers often promise to reduce or even eliminate compliance burdens through using their “certificate of compliance” programs without understanding or communicating the actual responsibilities and liabilities of a sponsor, manufacturer or CRO when using SaaS platforms. The sponsor or CRO is still responsible for completing over half of the common validation activities (e.g., validation planning, requirements development, GAMP® 5 CAT 4 and 5 functional testing, etc.) expected in a regulated platform.
In short, clinical research teams can be stuck with inadequate responsiveness or compliance strategies which put study outcomes at risk.
Change Controls and Responsibilities
Traditional validation also fosters the expectation that each software change must be strictly gated behind exhaustive document and assessment approvals and rigid change and configuration management processes. Requiring validated electronic signatures on all validation documents from multiple sources has become an expectation to minimize compliance risk even without any specific expectation or requirement from the regulatory guidance. Change and configuration management within software systems ensures that only authorized and approved updates are implemented in production systems; traditional IT procedures usually translate this to separating the role of the installer from developers, and traditional validation commonly enforces documenting and approving installation activities (IQ) before proceeding to test and validate other features.
Cost Constraints
With the activities described above, the time and effort required translate to significant ongoing costs for clinical sponsors and CROs. Using SaaS to reduce labor burdens only shifts some cost through the purchase of “validation kits” and templates from the SaaS provider. Activities require both IT and non-IT resources who are often tasked with competing priorities especially during study startup activities.
TRI Strategies for Adaptive Clinical Informatics
As a CRO with strong IT and clinical compliance services, TRI develops and hosts regulated systems for our clients and also evaluates and implements SaaS solutions from market leaders. We have identified many technical and procedural improvements to promote rapid delivery while also improving compliance. The following guiding principles drive many of our improvements:
- Value stream iterations
- Lean documentation generating validation outputs within 3-week sprint cycles
- Restructuring change control and validation plan processes less burdensome; “more open is not less secure”
- Automation
Embrace Frequent Value Stream Iterations
Sponsors and CROs need to get past the “buzz”. It is common to find teams or vendors who say they are “transformative” or “Agile” and use a lot of the right words. For example, one vendor promised they “do Agile” while they “still respect the needs” of regulatory authorities and compliance teams. Looking under the hood, this company like many others only used the terms and did the ceremonies (Agile term for meeting and processes) but did not actually get the customer engagement and continuous improvements expected in Agile. This often creates more friction within sponsors and IT teams as ceremonies seem to increase demands on staff but system quality and value remain stagnant. In short, Agile is not something that should be done halfway. Leadership and the organization commit to an Agile transformation and the values and principles explicitly implemented in Agile methodologies, including delivering frequently and at a constant pace while prioritizing responsiveness and value over rigid planning and documentation. Properly adopting iterative delivery and Agile ensures delivery on shifting study needs and rapid response while not compromising compliance, but it requires teams to radically revise the artifacts and processes they have used.
Adhere to Lean Documentation
Companies who have not embraced Agile or done so in name only are still spending a lot of effort creating, revising, and routing validation binder materials and supporting technical documentation. As Agile requires us to prioritize working software over comprehensive documentation, common Agile methodologies like Scrum do more than defer or reprioritize technical documents. Do not expect your Agile teams to generate traditional waterfall outputs and artifacts. Effective Agile teams focus on the software instead but still generate outputs that supports effective verification and software assurance activities. Some of the best practices within Agile include continuous documentation, test-driven design and an approach that documents software late in the lifecycle and only what is needed as opposed to other models which document everything early leading to delays and frequent changes to these binder artifacts. This is more of a culture change than an IT shift, as client and QA departments need to align expectations.
Fundamental in lean documentation is (a) a recognition that Agile practitioners (“agilists”) will still document software and (b) agilists will only document what is needed versus what is expected from older models. Though regulatory authorities like the FDA and EMA did not mandate specific documents or formats, industry has distilled the current guidance into the suite of plans, reports, matrices and other artifacts expected in traditional validation to the detriment of responsive, iterative delivery. The new guidances shift from Computer Systems Validation (CSV) to Computer Software Assurance (CSA). CSA provides more flexibility in the approach and acceptable records for regulatory authorities based on critical thinking, risk analysis and a focus on testing to demonstrate system quality and performance.
In short, Agile processes will include documentation of the backlog and user stories (requirements), traceability to testing, and independently verified outputs. These artifacts can replace more burdensome documents in traditional CSV. Agile teams can ensure that feature/story “risk” is explicitly captured to support the shift to CSA but are otherwise well-positioned to support CSA regulatory compliance following lean documentation principles.
Open Change Control and Validation Process
Older software lifecycle methodologies like Waterfall and traditional validation both contribute to gated, rigid and linear processes. Older methods are limited in accepting changes after moving into development; changes may need to wait months before the currently cycle delivers on the old requirements as opposed to the next 3-week spring. Teams cannot start development before the full requirements specification is approved and cannot deploy for testing or production releases until multiple iterations of reviews and signoffs are completed. Change and configuration management are, in short, not iterative or open.
Verification is still important in Agile and newer CSA approaches, but these flexible models allow teams to focus on delivery with less burdensome ceremonies. This shift is reflected in change control and validation processes. Modern techniques including Agile methodologies and Secure DevOps can allow Agile teams to move quickly through change control and configuration management through automation (below) and Continuous Integration/Continuous Delivery (CI/CD). Implementing CI/CD allows teams to rapidly move from delivery of one requirement to the next and breaks down some of the traditional silos and roles in the process. More open does not mean less secure; instead of relying on separate functional roles and “separation of duties” to maintain security, these techniques focus verifiable tests and interactions as gatekeepers of change and release management. Teams can deliver with less administrative burden without compromising quality or security. For regulated systems validation, this translates to incorporating the test and interactions as cornerstones of our evidence of “fit for intended use.”
As previously noted, the Agile is a cultural change as much as an IT transformation. In Agile and especially in the new CSA approach to compliance, non-IT stakeholders need to drive risk assessment for direct impact systems, ensuring that critical and high-risk feature requirements align with the CSA guidance for patient and product safety, quality, and data integrity. CSA moves from “validating everything” to verification based on risk. As a result, Agile and CSA require regular involvement from the clinical stakeholders (e.g., the Product Owner, Customer, etc.) when establishing the backlog and assessing risk through the acceptance of the released product. Teams can pivot from pausing to collect signatures on traditional binder artifacts to iterating through delivery with “just enough” documentation principles, but stakeholder involvement is still documented to demonstrate the product is viable and fit for use.
Automation
More than any other improvement in software lifecycle methodology or validation practice, automation is the single best driver for continuous compliance and efficiency in Agile/iterative clinical informatics. Automation produces consistent results and is often better evidence under regulatory inspection.
Traditional clinical GxP-relevant applications would be set up, validated once, and held in a steady state (unchanged) to minimize cost and risk. With research-driven needs for significantly higher rates of changes and iterations and keeping these systems in a continuously validated state, teams need to prepare and execute verification quickly. Automation in both testing and deployment drive the CI/CD process while simultaneously generating the output for CSA.
The requirements from the product backlog translate into tests; using test-driven design principles these are created and approved as part of development. Using tools like Selenium, teams create or update scripts to execute the different test expected (e.g., positive, negative, boundary tests, etc.) based on feature risk. As development completes, the automations verify the feature(s) and trigger delivery to integration testing and delivery processes, which themselves are automated through popular tools like JIRA and Azure DevOps. Automation, important for establishing and maintaining a reliable cadence and delivering the oft-quoted “Quality at Speed” objective of Agile, also delivers “Compliance at Speed” as the automated outputs are the evidence we need to show that.
Industry can provide regulatory inspectors and compliance auditors with stakeholder-approved backlog/story records and related test results without expending extraneous effort to repackage the same information in traditional validation binder artifacts. This is not achievable with the IQ/OQ/PQ formats expected though not mandated under the current traditional CSV approach, but using the native test results is expected and encouraged in emerging CSA techniques.
Automation does require a significant effort and investment as the automation itself requires validation, but the downstream efficiency cannot be overvalued. Quantifiable gains in both software delivery and compliance performance can be measured within: reliability of requirements and unit test coverage; shorter time to implementation; reduction in validation life cycle and documentation; reduction in broken build process; greater number of total tests executed as part of a build, etc. Statistical approaches should also be leveraged for different aspects like selecting test scope and data or root cause analysis consistent with FDA guidance for Process Validation. Ultimately this reinforces the GxP focus on patient and product safety, quality, and data security.
Conclusion
In drafting the guidance for new GxP-relevant platforms, the FDA cited concerns that “the industry lags in implementation of automated systems and new technologies due to the lack of clarity, outdated compliance approaches, and perceived regulatory burden.” They attempt to drive industry towards more responsive informatics techniques including an emphasis on less documentation and scripted, automated testing vs unscripted/manual validation. This now aligns with the software development industry and modern methodologies. Regulatory authorities still impose requirements, but this does not mean teams cannot release on cadence and on demand.
Clinical sponsors and CROs should embrace Agile solutions for clinical informatics in both CSV and new CSA approaches. This includes replacing the documents required in your current validation plans with the evidence and artifacts generated by testing and CI/CD automations and revisiting team structures to ensure stakeholders are recording risk inputs and approvals to illustrate verification towards intended use. At the same time, we need to be wary of clinical IT and compliance service providers who promote modern Agile and CSA services while still yoked to outdated process and document-focused delivery.
Over the next two years, our industry will move from CSV to CSA. IT teams and providers for sponsors and CROs should ensure they effectively use Agile techniques to make that transition a successful one.
About the Author
Matt Halnon TRI’s Director of Information Technology and Security, has been delivering IT systems for over 25 years. After a decade leading web and Java development teams, he moved into Clinical Trial and Life Science informatics with TRI in 2007 where he now sets strategy and vision for the CRO’s IT Division. Matt has a M.S. in Computer Science and membership with industry leaders including ISACA, the Agile Alliance, and the International Information System Security Certification Consortium (ISC2).
