By Nour Shahin
What happens when the tools that we trust to save lives unintentionally deepen the inequalities we’ve spent decades attempting to dismantle—or worse, perpetuate historical systems of oppression under the guise of innovation? As Artificial Intelligence rapidly advances and revolutionizes many aspects of public services in the UK, including the healthcare system, this question becomes increasingly urgent. At first glance, integrating AI into the healthcare system offers the promise of a significantly advanced and more efficient system, inspiring hope for a more progressive future. Nonetheless, a closer examination reveals a troubling reality beneath this veneer of progress: without effective regulation, these technologies risk perpetuating the very biases they aim to eliminate, echoing post-colonial patterns of exploitation and exclusion. Thus, as the UK government continues to develop a socially responsible AI framework, the healthcare system stands at a critical juncture: should it prioritize speed and innovation, or ensure that the systems are designed to promote fairness and equality? This debate constitutes the central focus of this article, beginning with an exploration of how AI has been integrated into the UK’s healthcare system, the current regulatory frameworks that govern its use, and the ethical and bias-related and post-colonial challenges that it presents. Finally, the article concludes with an emphasis that this not merely a matter of regulatory refinement, but rather a moral imperative that will ultimately define the trajectory of healthcare, and its impact on society.
The Integration of AI into the UK’s Healthcare System:
The vast and diverse applications of Artificial Intelligence (AI) within the UK’s healthcare system demonstrate its potential to transform medical services, while also exhibiting the complex challenges it engenders. From diagnostics to predictive analytics and administrative automation, AI is revolutionizing the delivery of healthcare services as we know them within the NHS [1].
One of the most significant ways AI has is reshaping healthcare provisions is through tools such as “C the Signs”, which introduce a more data-driven and systematic approach to early cancer detection [1]. This AI-powered tool utilizes sophisticated algorithms to analyse a patient’s symptoms and risk factors, providing clinicians with a probabilistic assessment of potential cancer diagnoses for the patient [1]. By facilitating earlier identification of potential cancer cases, tools like C the Signs have improved healthcare outcomes through enhanced diagnostic accuracy and timely intervention, collectively resulting in improved patient outcomes and increased survival rates. Additionally, AI integration has immensely progressed predictive analytics, especially regarding the detection of rare diseases and the analysis of genetic data. By leveraging algorithms to analyse coded data from electronic health records (EHRs), AI has enabled the identification of patterns indicative of genetic or rare conditions [1]. This proactive approach has allowed for earlier diagnoses, and thus, more targeted and precise interventions that improve patient outcomes and limit the need for complex late-stage treatments in the future.
Transcending diagnostics, AI has also played a crucial role in precision medicine by leveraging machine learning to predict the success of treatment protocols based on previous patient data [2]. This personalized approach allows for tailored interventions that not only improve clinical outcomes, but also reduce healthcare costs by avoiding the administration of ineffective treatments [2]. Moreover, in mental health care, AI systems support clinicians through the use of predictive models to analyse behavioural patterns, medical histories, and socioeconomic factors [3]. This has enabled clinicians to identify high-risk patients, and detect early warning signs of deterioration, ensuring timely interventions that significantly reduce hospital readmissions, improve continuity of care, and optimize resource allocation, so that mental health services are administered to those most in need [3].
Another way in which AI is being increasingly used is in the healthcare system is for administrative automation. AI technologies are now able to automate manual process, including capturing the details of patient consultations, and integrating decision-making outputs directly into electronic health systems [1]. These tools have aimed to enhance the healthcare system by attempting to alleviate the administrative workload of clinicians, and mitigating the time constraints associated with these tasks, allowing them to place greater focus on direct patient care. Moreover, clinical decision support systems, including AI-driven chatbots, for instance, are being more widely used by clinicians to assist in generating differential diagnoses and triaging patients [1]. The objective of these tools is to streamline clinical processes by proposing extensive diagnostic possibilities based on patient inputs, including symptoms and previous medical history, in order to improve diagnostic accuracy and efficiency.
Ultimately, these advancements demonstrate how AI is significantly transforming the UK’s healthcare system, promoting innovation in diagnostics, predictive analytics, administrative processes and clinical decision-making. However, adjacent to these revolutionary advances lies the critical question regarding their implementation and regulation: whether the purported benefits of these technologies truly outweigh the potential challenges, or if these risks are not only overlooked but also perpetuate historical inequalities and post-colonial systems of power. Furthermore, it remains unclear if the existing regulatory frameworks can efficiently foster a balance between innovation and ethics—or if such a balance can even be achieved in practice, given the entrenched structure of bias and global inequities these technologies may inadvertently sustain.
The Regulatory Frameworks Governing AI In UK Healthcare:
This leads us to the pressing question of how AI is currently regulated within the UK’s healthcare system, and whether these frameworks are effective in fostering a balance between the rapid pace of technological innovation and the ethical safeguards needed to ensure equitable and just implementation.
The General Data Protection Regulation (GDPR) forms the cornerstone of the UK’s regulatory landscape, providing a robust framework that ensures patient data is ethically and securely administered in AI healthcare applications [4]. By mandating principles such as data minimization, purpose limitation and explicit consent, the GDPR promotes accountability and transparency in managing sensitive health data 4. Furthermore, the GDPR requires “privacy by design”, a concept which embeds data protection throughout the entire development and operationalisation of AI systems to ensure the protection of individual privacy rights and the build of public trust [4].
Alongside the GDPR, the UK Government’s National AI strategy outlines a ten-year vision intended to advance AI development whilst simultaneously upholding standards of ethical integrity, safety and accountability [5]. Foundational to this strategy are initiatives such as the Centre for Data Ethics and Innovation (CDEI), and NHS England Guidelines, which emphasise transparency, safety and trustworthiness in AI’s development and deployment [5]. Nonetheless, despite these efforts, significant gaps still remain in the regulation of AI within the UK’s healthcare system, especially surrounding the mitigation of algorithmic bias—a crucial challenge that threatens to diminish the equitable delivery of medical care.
As inferred by the aforementioned regulations that have been put in place, the UK government has evidently emphasized safeguarding patient rights, such as privacy and the ethical integration of AI into healthcare system. However, these regulatory frameworks disregard the unsettling reality that AI systems risk perpetuating and intensifying these biases, leading to marginalized populations being disproportionately disadvantaged [4]. The diagnostic algorithms that underpin AI systems are often trained on predominantly homogenous datasets that reflect historical and systemic inequalities, leading to less accurate results for underrepresented groups, and thus exacerbating health disparities [4]. Furthermore, this problem is further complicated by the “black-box” nature of several AI systems, in which the decision-making processes are opaque, and therefore, difficult to interpret [6]. This lack of transparency obscures the inner workings of algorithms, making it near to impossible to identify, and address, embedded biases that disproportionately affect marginalized groups [6]. Without processes put in place to enhance accountability or explainability, biased outcomes remain unchallenged and, quite disconcertingly, the standard practice. This allows these AI tools to reinforce systemic inequities in healthcare, rather than aid in dismantling and
eliminating them.
This regulatory failure to account for demographic diversity in AI training raises a pressing question: do these systemic oversights within current regulatory frameworks inadvertently perpetuate pre-existing socio-demographic biases? In addition, does this fundamentally align AI with post-colonial structures of exploitation, in which marginalized communities continue to face inequalities under systems built by, and built for, dominant power structures?
Algorithmic Bias in Skin Cancer Detection: A Case Study of Inequities in UK Healthcare:
The inherent structural biases embedded within our societies have manifested in AI systems used in public services. This is starkly illustrated by the example of skin cancer detection algorithms in the UK. A 2021 study highlighted the critical issue of extreme underrepresentation of darker skin tones in training datasets, and its impact on diagnostic accuracy [7]. These algorithms are often reliant on convolutional neutral networks (CNNs), and are primarily trained on datasets composed of light-skinned individuals, with Black patients comprising as little as 5-10% of the training data included [7]. This critical underrepresentation in datasets has resulted in negative repercussions: the diagnostic accuracy for Black patients is approximately half of that for white patients, revealing the failure of these AI systems to perform equitably and with improved efficiency. Rather, their exacerbation of healthcare inequalities systemically disadvantages already marginalized communities [7].
The impact of this bias is evident in melanoma outcomes in the UK: Black patients face significantly higher mortality rates from melanoma, with a 5-year survival rate of merely 70%, compared to the 94% for white patients [7]. This is further compounded by the failure of AI-driven diagnostic tools to accurately detect early-stage melanomas in darker skin tones, which results in delayed diagnoses, and thus, advanced cancer stages at the time of detection [7]. Moreover, these delays are highly likely to reduce the availability of effective treatment options, leading to disparate outcomes for particular racial groups within healthcare systems that are ostensibly designed to be equitable [7].
Nevertheless, the use of non-representative training datasets in AI-driven tools in the case of skin cancer detection algorithms is emblematic of a wider structural complexity within the development of AI systems and tools—an apparent indifference to diversity that prioritizes dominant demographic groups whilst ignoring others. This emphasizes a fundamental gap in regulatory frameworks, suggesting that they privilege and prioritize efficiency and speed in the development of these tools, rather than seriously considering their consequences. This poses the urgent question of whether AI tools inadvertently reinforce post-colonial structures of exclusion, perpetuating systems that favour the historically advantaged while marginalizing underrepresented populations [7]. Ultimately, this reveals both a critical tension; the very tools that are intended to revolutionize healthcare, are amplifying societal inequalities.
The Post-Colonial Dilemma: A Challenge Beyond Regulation?
This disconcerting ethical failure urges us to confront a vital question: are these AI-driven innovations truly a pathway to equitable progress, or are they, reinforcing the injustices they claim to dismantle under the guise of technological innovation? And if the latter, is there a genuine possibility for regulatory frameworks to address these deeply embedded inequalities, or do they merely serve to conceal the existing structural imbalances present in our society?
These structural biases embedded within AI systems transcend merely their applications in healthcare, presenting AI as a manifestation of broader historical and systemic inequities, deeply-rooted in post-colonial power structures. Decolonial theorists argue that AI, as a product of our vastly technological society, exacerbates the patterns of exclusion established during colonialism, by virtue of its reliance on datasets that prioritize Western-centric knowledge systems and its disregard to diverse local contexts [8]. In regards to the healthcare system, as aforementioned, the underrepresentation of minority demographic groups in training datasets for AI tools perpetuates health disparities instead of attempting to rectify them. This pattern of algorithmic bias not only reflects historical racial hierarchies, but also reconstructs them into modern digital infrastructures, continuing to marginalize communities that are already fundamentally disadvantaged by pre-existing systemic inequities [9]. The implications that these biases perpetuate are profound, deeply intertwined with the structural inequities in public health policies and practices. The global dominance of AI technologies, predominantly designed and controlled by a few economic superpowers, marginalizes the voices and needs of developing regions, a phenomenon described as “digital colonialism” [9]. Within the UK, these dynamics exist through the prioritization of “value for money” algorithms, which inadvertently exclude vulnerable populations from accessing equitable healthcare. By virtue of emphasizing cost efficiency over inclusivity, this approach reinforces a neo-colonial reasoning that commodifies human experiences and bodies for profit [9]. Embedded within the broader framework of “algorithmic dispossession”, these systems systematically divert resources and opportunities away from marginalized communities [9].
As explored within this article, efforts to regulate AI in healthcare, such as the GDPR and the UK’s National AI Strategy, fail in addressing these deep-rooted issues of inequality. Albeit these frameworks highlight the significance of adhering to principles of privacy and transparency whilst utilizing AI tools, they fall short in interrogating the colonial legacies that underpin the development and deployment of AI systems [8]. This lack of critical engagement perpetuates a “universalist” approach to ethics that completely disregards the socio-political contexts in which these tools operate, thereby sustaining a hierarchical global order wherein the benefits of AI are
accessible to a privileged few [9].
Ultimately, the question lies in whether these post-colonial challenges can be effectively deconstructed by means of regulation alone, or whether they represent the insurmountable challenge of a rapidly digitized world deeply entangled in historical systems of exploitation and exclusion.
Conclusion: The Unresolved Dilemma of AI in Healthcare
The integration of AI into healthcare systems in the UK has demonstrated its potential, and its perils. While AI has driven significant advancements in diagnostics, predictive analytics, and personalized medicine, these advantages are greatly overshadowed by its significant challenges. Bias in algorithms, deep-rooted in historical inequalities and homogenous datasets, exhibit how these tools risk perpetuating systemic inequalities rather than dismantling them, which has been evidenced by skin cancer detection algorithms used in the NHS.
Furthermore, regulatory frameworks, such as the GDPR and the National AI strategy, albeit foundational, fall short of addressing these deeper systemic problems. They place immense focus on transparency and privacy, yet fail to address the colonial legacies and global power imbalances embedded within AI systems. As such, they raise profound questions regarding the role of regulation and innovation in mitigating AI’s risks. Can enhancing regulation truly eliminate these deeply entrenched biases, or are these inequalities an inevitable by-product of an increasingly digitized world shaped by our pre-existing power hierarchies? Do we have the ability to redirect AI development toward inclusivity, or will these tools always reflect and amplify societal inequalities under the guise of progress and advancement?
Therefore, as we consider the future of artificial intelligence in healthcare, this critical question remains: is there a viable pathway to decolonize AI, or does the promise of AI as a transformative tool for progress ultimately fall short, leaving the most vulnerable to bear the brunt of its systemic failures? The resolution to this dilemma lies in more than just technological refinement—it requires confronting the moral and structural challenges of AI in a world where innovation must balance progress with justice.
Bibliography
RazaiI, M. S., Al-bedaery, R., Bowen, L., Yahia, R., Chandrasekaran, L., & Oakeshott, P. (2024, November 21). Implementation challenges of artificial intelligence (AI) in primary care: Perspectives of general practitioners in London UK. PLoS ONE, 19(11).
Dicuonzo, G., Donofrio, F., Fusco, A., & Shini, M. (2023). Healthcare system: Moving forward with artificial intelligence . Technovation.
Dawoodbhoy, F. M., Delaney, J., Cecula , P., Yu , J., Peacock, I., Tan, J., & Cox , B. (2021). AI in patient flow: applications of artificial intelligence to improve patient flow in NHS acute mental health inpatient units . Heliyon.
Morley, J., Machado, C., Burr, C., Cowls, J., Joshi, I., Taddeo, M., & Florid, L. (2020). The ethics of AI in health care: A mapping review. Social Science & Medicine.
McGrath, C. P. (2024). The state and values of AI governance in UK healthcare. In B. Solaiman, & I. Cohen, Research handbook on health, AI and the law. Edward Elgar Publishing .
Floridi, L., & Cowls , J. (2019). A Unified Framework of Five Principles for AI in Society. Harvard Data Science Review .
Norori, N., Hu , Q., Aellen, F., Faraci, F., & Tzovara, A. (2021). Addressing bias in big data and AI for health care: A call for open science . CellPress.
Adams, R. (2020). Can artificial intelligence be decolonized? . SAGE Journals .
Mohamed, S., Png , M.-T., & Isaac , W. (2020). Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence . Philosophy & Technology .
So insightful