A comprehensive overview of three essential aspects of computational modeling: code verification results, model calibration evidence, and bench test validation results.
Each of these plays a vital role in ensuring the accuracy and reliability of computational models in various scientific and engineering domains.
Table of content
A comprehensive overview of three essential aspects of computational modeling: code verification results, model calibration evidence, and bench test validation results.
Each of these plays a vital role in ensuring the accuracy and reliability of computational models in various scientific and engineering domains.
The Food and Drug Administration (FDA or the Agency), the US regulating authority in the sphere of healthcare products, has published a guidance document dedicated to the processes and procedures associated with assessing the credibility of computational modeling and simulation in medical device submissions.
The document provides an overview of the applicable regulatory requirements, as well as additional clarifications and recommendations to be taken into consideration by medical device manufacturers and other parties involved to ensure compliance to them.
At the same time, provisions of the guidance are non-binding in their legal nature, nor are they intended to introduce new rules or impose new obligations.
Moreover, the authority explicitly states that an alternative approach could be applied, provided such an approach is in line with the existing legal framework and has been agreed with the authority in advance.
The document describes, inter alia, the aspects related to credibility evidence to be used to support claims made by medical device manufacturers concerning the safety and effectiveness of the products they are responsible for.
Code Verification Results
According to the guidance, the process of code verification is crucial in computational modeling as it provides evidence that the software accurately implements the underlying mathematical models and numerical algorithms.
This step is fundamental to ensure that there are no bugs in the software that could affect the numerical accuracy of simulations.
One common approach in code verification involves comparing the outcomes of the computational model against analytical solutions, which are often derived using methods like the method of manufactured solutions.
An essential aspect of this comparison is the verification that errors in the computational model decrease as expected when the sizes of spatial and temporal discretizations are reduced.
This process is particularly important in fields such as solid mechanics, fluid dynamics, heat transfer, and electromagnetism, where models frequently rely on solving complex partial differential equations.
Model Calibration Evidence
Model calibration involves adjusting model parameters to minimize the discrepancy between the model’s results and the calibration data.
This process, however, is distinct from validation as it does not involve testing the final model against independent data.
Calibration is an assessment of how well the simulation results fit with the data used.
The authority additionally emphasizes that even though calibration evidence is considered weaker than validation evidence, it still plays a crucial role in supporting the credibility of the model.
Robust calibration evidence is especially compelling when a model can reproduce complex behaviors by adjusting only a few parameters, based on fundamental principles rather than solely on the calibration data.
An example of this can be seen in physiological modeling, where patient-specific models of heart functions are calibrated to match clinically measured data.
Another example is in heat transfer modeling, where models are calibrated to replicate in vivo tissue heating patterns by adjusting parameters like the blood-tissue heat transfer coefficient.
Bench Test Validation Results
Bench test validation involves using experimental data from controlled laboratory settings, distinct from clinical or animal testing.
This type of testing offers a well-controlled environment, advantageous for validating simulations. Bench testing can encompass various non-clinical tests, including in vitro and cadaveric tests.
The results from these tests can be supported by calculation verification and Uncertainty Quantification (UQ) results from the validation simulations.
Bench test validation can be structured in different ways.
It could involve prospectively planned validation activities, validation against retrospective experimental datasets, or use previously generated validation results.
In solid mechanics, for instance, manufacturers of cardiovascular implants might use computational models to predict the durability of new devices, comparing these predictions with actual bench test results.
In electromagnetics, bench testing might involve comparing computational predictions of energy absorption during MR scans with physical experiments using similar devices in controlled environments.
Furthermore, the document discusses the comparison of validation cases based on the planning of validation simulations and comparator data.
When validation simulations are prospectively planned, there is a greater degree of control over the relevance to the context of use (COU) and the quantification of uncertainties in the simulation results.
However, when using retrospective comparator data, the control over the relevance and credibility of the validation activities might be limited.
In cases where previously generated validation simulations are used, the ability to tailor experiments and simulations to the current COU is often restricted, leading to potential challenges in ensuring the applicability and credibility of the validation.
Conclusion
In summary, the present FDA guidance underscores the nature of computational modeling, highlighting the importance of thorough verification, calibration, and validation in various scientific and engineering fields.
These processes not only ensure the accuracy and reliability of computational models but also contribute significantly to their credibility and practical applicability in diverse real-world scenarios.
How Can RegDesk Help?
RegDesk is a holistic Regulatory Information Management System that provides medical device and pharma companies with regulatory intelligence for over 120 markets worldwide. It can help you prepare and publish global applications, manage standards, run change assessments, and obtain real-time alerts on regulatory changes through a centralized platform. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Global expansion has never been this simple.
