Bias in Medical Device Research: New Considerations for PMCF Surveys
By Hemant Mistry and Matthew Hanson
Identifying the Issue
Recent evidence has increasingly pointed to racial and ethnic biases in the performance of certain medical devices widely used in the NHS across the UK.
In post-approval studies of medical devices, it was found that:
Only 14% included gender as a key outcome measure
A mere 4% conducted subgroup analyses for female participants
Given the NHS's fundamental duty to ensure the highest standards of safety and effectiveness for all patients, such biases result in suboptimal treatment for affected population groups.
UK Government Supported Review and Task Force Initiative
In response to growing concerns about bias in medical device design and research, a government-supported review was published in March 2024. This comprehensive review aimed to identify potential biases in medical device development and usage. Titled "Addressing Bias in Medical Devices," the review sought to uncover areas where racial, ethnic, and gender biases might exist.
To tackle these issues, the review proposed the establishment of a dedicated task force. This task force, which includes the Medicines and Healthcare products Regulatory Agency (MHRA) and other key stakeholders, is charged with implementing a robust action plan. The goals of this plan are to:
Anticipate Potential Harm: Proactively identify and mitigate risks associated with biased medical devices.
Reduce Bias: Develop strategies to minimize biases in both the design and application of medical devices.
Create Impactful Recommendations: Formulate and disseminate recommendations that address medical device inequity, ensuring fair and effective treatment for all patient groups.
This initiative underscores the commitment to maintaining the highest standards of safety and effectiveness in medical devices, ultimately striving for equitable healthcare outcomes for all.
Research Findings
The research revealed several critical issues, for example:
Pulse Oximeters: These devices overestimated the oxygen levels in the blood of individuals with darker skin tones.
AI-Enabled Devices: These showed a clear bias against women, ethnic minorities, and disadvantaged socioeconomic groups.
These groups are often underrepresented in clinical trials and device evaluations, leading to the underdiagnosis of medical conditions. Without appropriate action, such biases will persist throughout the medical device lifecycle—from research and development to approval, deployment, and post-market monitoring, as well as in the actual use of these devices.
Recommendations
The review emphasised the need for a greater focus on preventing adverse events in new devices through proactive and continuous collection and evaluation of safety and performance data, particularly across diverse subgroups (e.g., by skin tone). This approach ensures that any adverse outcomes are identified early and mitigated effectively.
Impact
The review has had a significant impact, with the MHRA now requiring equity assessments as part of the approval process for new device applications and in post-marketing surveillance. This ensures that medical devices are safe and effective for all patient groups, promoting equitable healthcare outcomes.
Biases that can apply to Post Market Clinical Follow Up (PMCF)
Within the medical device space, and its usage in the clinical setting, bias can exist in three main forms:
Interpretation
Created through the user (e.g., when clinicians apply unequal, race-based standards to the readouts from medical devices and tests)Computational
Lies within the software or in the data sets used to develop the gadget. Example of where this may be evident are in cases where medical devices are tested primarily on a homogeneous group of subjects (typically white males)Physical
Found inherently within the mechanics of the device
With PMCF data collection being an essential part of EU-MDR compliance and as MDCG 2020-7 cites that within the PMCF Plan there must be ”Justification of the study design on the basis of all of the above, and why it is sufficient to ensure representative patient populations and provide for adequate controls on sources of bias”, it is important to consider potential sources of bias.
Both Computational and Interpretation bias can be encountered during PMCF data collection, with the main threat at both ends of the project lifecycle; material design and reporting of results. For example:
Selection bias: Survey participants aren’t representative of all users, as quotas for gender/ethnicity for example don’t tend to be included
Response bias: Badly designed questions may lead to skewed answers
Recall bias: Respondent recollection of past experiences can be inaccurate
Reporting bias: Selective revealing or suppression of information
Observer bias: Data collectors’ preconceived ideas can influence results
How is Purdie Pascoe addressing these issues?
Having conducted over 400 PMCF Surveys over the last 5 years, Purdie Pascoe has expertise in survey planning and design and can provide valuable context and data to minimize bias. We specialise in identifying significant imbalances and trends in PMCF surveys, bringing these insights to our clients' attention and encouraging their application. This proactive and supportive approach enables us to showcase our core values in real-world practice: Supportive, Specialist, Committed, Bold.