The present article details FDA guidance on software testing, including tests performed by the manufacturer (software developer) regarding potential changes to the software. The basics of software testing are described in the initial article.
The Food and Drug Administration (FDA or the Agency), the US regulating authority in the sphere of healthcare products, has published a guidance document dedicated to the general principles of software validation. The document provides an overview of the applicable regulatory requirements and also highlights the most important aspects to be considered by medical device manufacturers and other parties involved. It is also important to mention that due to its legal nature, the guidance is not intended to introduce new rules and obligations, but rather to provide additional clarifications and recommendations regarding the way compliance with existing legislation could be achieved and sustained. Moreover, an alternative approach could be applied, provided such an approach complies with the respective regulatory requirements and has been agreed with the authority in advance.
Table of Contents
Testing of Software Changes
The document pays special attention to the testing of software changes. The authority acknowledges that medical device manufacturers (software developers) may introduce changes to the product from time to time to improve its overall performance and/or safety. As it is further explained by the FDA, these changes are the results of:
- Debugging that finds an error and it is corrected;
- New or changed requirements (“requirements creep”); and
- Modified designs as more effective or efficient implementations are found.
According to existing FDA requirements, once software has been approved for marketing and use, any further changes should be subject to rigorous assessment to identify the way they will impact the safety and performance of the product, and this assessment includes software testing as well. The latter should demonstrate that (a) the changes have been implemented perfectly, and (b) the changes would not harm other components. For this purpose, regression analysis and testing should be undertaken. As it is explained by the Agency, regression analysis is the determination of the impact of a change based on the review of the relevant documentation (e.g., software requirements specification, software design specification, source code, test plans, test cases, test scripts, etc.) to identify the necessary regression tests to be run; while regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current results to the previous result to detect unintended effects of a software change. Both the analysis and testing described hereinabove should be applied to ensure the changes the software product was subject to would not impact adversely the overall performance of the product, as well as the performance of existing components.
Testing Levels
The FDA guidance also describes in detail the way the testing could be divided into levels. According to the guidance, the testing could be undertaken at four levels: unit, integration, and system respectively. The document outlines the following specific features of each of the testing levels:
- Unit (module or component) level testing focuses on the early examination of sub-program functionality and ensures that functionality not visible at the system level is examined by testing. This testing level is intended to ensure that any components of the software meet the respective requirements in terms of quality.
- Integration level testing focuses on the transfer of data and control across a program’s internal and external interfaces. Hence, this testing level deals with the aspects related to the way the software interacts with other products (both software and hardware).
- System-level testing demonstrates that all specified functionality exists and that the software product is trustworthy. At this level, the overall performance of the product should be assessed concerning the initial requirements. According to the document, this covers the following aspects:
- Performance issues;
- Responses to stress conditions, e.g., behavior under maximum load, continuous use;
- Operation of internal and external security features;
- Effectiveness of recovery procedures, including disaster recovery;
- Usability;
- Compatibility with other software products;
- Behavior in each of the defined hardware configurations; and
- Accuracy of documentation.
The authority additionally emphasizes the importance of ensuring that such testing covers any significant matters.
According to the FDA guidance, the way the product operates in its intended use environment (e.g., clinical setting) should also be assessed in the context of system-level testing. Thus, to ensure that the results of such testing would be accurate and reliable, the medical device manufacturer (software developer) should reconstruct the environment in which the product in question is intended to be used. For instance, it could be an option to undertake a simulation and/or test customer locations. Furthermore, the respective testing plans should also describe specific controls and measures to be duly implemented to ensure that testing is performed properly, and its results are duly documented. In this regard, the authority additionally emphasizes that the sites whether the testing takes place should not be directly controlled by the software developer. It is also important to mention that for a software product that is a medical device or a component of a medical device that is to be used on humans before FDA clearance, testing involving human subjects may require an Investigational Device Exemption (IDE) or Institutional Review Board (IRB) approval.
Additional Considerations
About the test procedures and results the authority states that they should be duly documented. In particular, it should be indicated whether the applicable pass/fail criteria have been achieved. Apart from the final results, any decision-making process related to the software testing should be documented as well, and such documents should be provided for review upon request. Any issues identified in the course of testing should be also duly recorded and analyzed to assess the way they would impact the overall safety and effectiveness of the product. To ensure consistency, test reports should be in line with the initial requirements prescribed by the test plans.
The Agency also acknowledges that in the course of the software development additional software tools could be used. In this respect, the authority states that such tools should have a degree of quality no less than the software product they are used to develop.
In summary, the present FDA guidance provides additional clarifications regarding such important aspects related to software testing as change testing and testing levels. The document clarifies the scopes of testing levels to be applied, and the particular matters to be assessed at each of the levels.
Sources:
How Can RegDesk Help?
RegDesk is a next-generation web-based software for medical device and IVD companies. Our cutting-edge platform uses machine learning to provide regulatory intelligence, application preparation, submission, and approvals management globally. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Applications that normally take 6 months to prepare can now be prepared within 6 days using RegDesk Dash(TM). Global expansion has never been this simple.
