The article describes in detail the approach to be applied for credibility factors and credibility goals to be considered by the parties responsible for medical devices.
Table of content
The Food and Drug Administration (FDA or the Agency), the US regulating authority in the sphere of healthcare products, has published a guidance document dedicated to assessing the credibility of computational modeling and simulation in medical device submissions.
The document provides an overview of the applicable regulatory requirements, as well as additional clarifications and recommendations to be taken into consideration by medical device manufacturers and other parties involved to ensure compliance with them.
At the same time, provisions of the guidance are non-binding in their legal nature, nor are they intended to introduce new rules or impose new obligations.
Moreover, the authority explicitly states that an alternative approach could be applied, provided such an approach is in line with the existing legal framework and has been agreed with the authority in advance.
The scope of the guidance covers, inter alia, the aspects related to credibility factors and credibility goals.
Credibility Factors and Credibility Goals
According to the guidance, in Step 5 of the framework, the focus is on defining credibility factors for planned credibility evidence and setting specific goals for each factor, along with a strategy to achieve these goals.
The ASME V&V 40 standard is a key reference for understanding credibility factors. It categorizes credibility factors into various aspects like “Software Quality Assurance”, “Numerical code verification”, “Calculation verification”, “Validation”, and “Applicability”.
To effectively establish these factors and goals, a series of sub-steps are recommended:
- Identification of Credibility Factors
The first sub-step, 5.1, involves stating credibility factors relevant to the type of credibility evidence planned for collection.
The ASME V&V 40 factors are advised as a starting point. In situations where the evidence categories are not covered by ASME V&V 40, such as model calibration or population-based evidence, the creation of new credibility factors is suggested. - Utilization of Credibility Factors
In cases where multiple forms of evidence are used, with one being the primary source and others as secondary, it’s recommended to apply ASME V&V 40 factors for the primary evidence and a limited set for the secondary evidence. This approach helps in managing the total number of factors and maintaining focus. - Relevance to Context of Use (COU)
Special emphasis is given to the factor of “Relevance to the COU”, highlighting the importance of evidence in supporting the model’s application within its intended context. - Grading of Activities
In sub-step 5.2, a gradation of activities for each credibility factor is defined following ASME V&V 40. This gradation represents different levels of rigour in the investigation process. - Selection of Credibility Goals
Sub-step 5.3 entails selecting a “credibility goal” from the activity gradation, considering the model risk assessed earlier.
The goal should align with the risk level associated with the model. If the goal is less than the risk level, a justification for the sufficiency of the activities is required. Additionally, a high-level plan to achieve the proposed credibility goal is necessary.
Adequacy Assessment
- Prospective Adequacy Assessment (Step 6)
This step involves evaluating if the credibility goals, once achieved, will make the evidence sufficient to support the model’s use for the COU considering the risk assessment.
As explained by the authority, this assessment is crucial, especially if seeking FDA feedback on planned activities. It includes analyzing the planned credibility evidence, proposed goals, and any other relevant information. - Post-Study Adequacy Assessment (Step 8)
This step re-evaluates the credibility level achieved for each factor and whether the credibility goal was met.
It includes a rationale explaining why the evidence supports using the model for the COU given the risk assessment. This assessment may also consider COU simulation results and related information, like the proximity of model predictions to safety thresholds. - Decision-Making Based on Adequacy Assessment:
According to the document, questions concerning the comprehensive testing of the model, the achievement of credibility goals, and the proximity of model predictions to decision or safety thresholds are vitally important. Recommendations include pre-specifying quantitative accuracy targets and considering uncertainty estimates in model predictions. - Addressing Insufficient Evidence
In cases where the evidence is deemed insufficient, whether in prospective or post-study assessments, options include modifying the model, altering the COU, or revising the plan to generate or collect additional credibility evidence.
Conclusion
In summary, the steps outlined in the framework emphasize the importance of carefully defining and assessing credibility factors and goals about the model’s context of use.
They guide the process of ensuring that the credibility of a computational model is adequately established and maintained throughout its lifecycle, especially in contexts of regulatory compliance.
How Can RegDesk Help?
RegDesk is a holistic Regulatory Information Management System that provides medical device and pharma companies with regulatory intelligence for over 120 markets worldwide. It can help you prepare and publish global applications, manage standards, run change assessments, and obtain real-time alerts on regulatory changes through a centralized platform. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Global expansion has never been this simple.
