The new article addresses the aspects related to the Modification Protocol section, especially to its content in general and data management practices in particular.
Table of Contents
The Food and Drug Administration (FDA or the Agency), the US regulating authority in the sphere of healthcare products, has published a draft guidance document dedicated to the regulatory requirements for a predetermined change control plan (PCCP) to be included in marketing submissions for an artificial intelligence/machine learning (AI/ML)-enabled device software functions (ML-DSFs). The document provides an overview of the applicable regulatory requirements set forth under the existing framework, as well as additional clarifications and recommendations to be taken into consideration by medical device manufacturers and other parties involved in order to ensure compliance thereto. At the same time, provisions of the guidance are non-binding in their legal nature, nor are they intended to introduce new rules or impose new obligations. Moreover, an alternative approach could be applied, provided such an approach is in line with existing legislation and has been agreed with the authority in advance.
The concept of a PCCP is intended to reduce the regulatory burden and facilitate further development of medical devices utilizing novel technologies while ensuring their safety and proper performance. Under the general rule, the changes reflected in a PCCP included in the initial application would not require a new marketing submission before being implemented.
The document describes in detail the requirements a PCCP should meet, including its main components. As explained by the authority, a PCCP should include, inter alia, a Description of Modifications and a Modification Protocol.
Modification Protocol: Key Points
According to the guidance, there are four main components of a Modification Protocol, each of them describing the manufacturer’s (1) data management practices, (2) re-training practices, (3) performance evaluation protocols, and (4) update procedures, including communication and transparency to users and real-world monitoring plans, for each modification in a PCCP. As explained by the authority, the abovementioned elements would provide the information sufficient for the authority to proceed with reviewing the plan in question. At the same time, in specific cases, additional information could be required in order to ensure that important matters related to safety and performance are addressed properly.
The particular methods to be applied would depend on the nature of the modifications and risks associated thereto. However, the authority encourages the applicants to ensure all the abovementioned components are in place with respect to each of the modifications described in a Modification Protocol. Apart from that, the applicant should also provide an explanation of how the methods used are similar to or different from the other methods used within the submission. According to the example provided by the authority, if the validation methods in the Modification Protocol represent a subset of the original testing for the device, or if the acceptance criteria for the validation are different, manufacturers should describe these differences and provide a justification; the justification for a different methodology may include references to other marketing submissions where the methodology was used for similar modifications.
The authority also emphasizes the importance of following the risk management processes when developing a Modification Protocol. Should the authority identify that not all of the modifications described in a PCCP are duly reflected in a Modification Protocol, the missing ones would not be included in an authorized PCCP and, consequently, require a new submission to be implemented. Thus, it is vitally important for the applicant to ensure a Modification Protocol covers any and all modifications intended to be included in a PCCP; otherwise, they will be excluded from it.
Data Management Practices
The document further provides additional clarifications and recommendations regards the content of the Modification Protocol section to be included in a PCCP. In particular, the authority explains how each of the four abovementioned main elements should be addressed.
The first element addressed in the guidance relates to data management practices. Training and testing a medical device based on a machine learning technology usually require specific data to be used to train the algorithm and the appropriate reference standard. In this respect, special methods are to be used to ensure the device is trained properly and identify and eliminate potential bias impacting the performance of the device. Proper training improves the overall accuracy and reliability of the information the device provides when used for its intended purpose. Due to the nature of ML-based medical devices, new data is required to support their continuous training. According to the guidance, the data management practices in a Modification Protocol should outline how those new data will be collected, annotated, curated, stored, retained, controlled, and used by the manufacturer for each modification; the data management practices should also clarify the relationship between the Modification Protocol data and the data used to train and test the initial and subsequent versions of the ML-DSF. Apart from that, details should also be provided on the methods used to curb the potential for data or performance information leaking into the development process during modification development or assessment.
The information outlined herein would be necessary for the authority to assess the data management practices employed by the medical device manufacturer. In particular, the aspects the authority will pay the most attention to are:
- The way the manufacturer obtains and uses the data while ensuring it is complete and representative in terms of the intended use population;
- Whether the subpopulations are also taken into consideration;
- The way training and testing data is sequestered;
- The way older data is replaced by newer data in order to ensure the continued effectiveness of a medical device;
- Whether the standards the medical device manufacturer is referring to are the most appropriate ones;
- The way the practices implemented may reduce the risks associated with discriminatory outcomes.
In summary, the present FDA guidance describes the approach to be applied by the applicants with respect to the content of a Modification Protocol and key elements to be included therein. The authority provides an overview of the applicable requirements and explains the importance of addressing data management policies.
How Can RegDesk Help?
RegDesk is a holistic Regulatory Information Management System that provides medical device and pharma companies with regulatory intelligence for over 120 markets worldwide. It can help you prepare and publish global applications, manage standards, run change assessments, and obtain real-time alerts on regulatory changes through a centralized platform. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Global expansion has never been this simple.
