By Hassan Raza, Senior Associate, Esquare Legal
There is a quiet change taking hold which you notice when entering the Services Hospital in Lahore on any day in the scorching heat of June, looking at the overburdened public hospitals of Johannesburg or the rural health outposts of Uttar Pradesh in India. This is not linked to a new kind of medicine or some new practice, but the use of AI models capable of scanning X-rays, drafting histopathology reports, and helping doctors in writing clinical notes.
It helps us realise that Artificial Intelligence is not a luxury but an advancement that can help bridge the gap between clinical demand and infrastructural scarcity. It is my belief that the success of medical AI will not be determined only by the usage of its code, but by the effectiveness of the legal frameworks we build to govern it.
The Promise and the Gap
The promise is undeniable. AI can increase the clinical capacity where specialists do not exist and can help ensure resource allocation in a world where medical supply chains remain disrupted. But the rapid deployment of these technologies at times outpaces the evolution of law. The LMICs — low and middle income countries — with their limited resources face a dual crisis: a severe shortage of medical professionals and a divided health data ecosystem.
If the space is not regulated by those who are aware of the challenges, then the adoption of medical AI can introduce fatal risks. Algorithmic bias based on the kind of data systems have been exposed to, and the compromise of sensitive personal health data, are not hypothetical concerns. There is a need to stop viewing the law or regulations as restrictive barriers and ensure that they become a critical enabler of innovation.
The Rights Framework
It is the right time for the conversation to start in developing states like South Africa and Pakistan. The right to health is not merely a policy goal but is a fundamental part of constitutional practice. South Africa’s Section 27 and Pakistan’s Article 9 (the right to life) make it obligatory that the state ensure equitable, transparent, and accessible healthcare. AI-powered diagnostics for tuberculosis or breast cancer can materially improve outcomes, and their deployment falls within the protective ambit of these fundamental rights.
The Data Equity Problem
The AI tools being deployed in LMICs are trained on datasets from high-income countries while systematically excluding local genetic, environmental, and cultural variances. This is not just a technical flaw but a result of non-neutral interventions in healthcare. If an algorithm trained in the West fails to recognise pathogenic patterns in a South American population, that encodes discrimination. Legally compelling algorithmic equity using national laws and international instruments is not an aspirational goal — it is a low-hanging fruit that remains underutilised in international law.
The Liability Question
The current jurisprudence often relies on the human-in-the-loop fiction. The physician remains the sole liable partner, which is inherently unjust. It would be unwise to expect a rural practitioner to override a statistically superior machine — this ignores the reality of automation bias entirely. There is a need to restructure liability toward enterprise and product liability models. LMICs should explore state-backed no-fault compensation funds akin to vaccine injury programmes. This would ensure swift justice for injured patients while shielding clinicians from innovation-averse litigation.
Building the Right Framework
The integration of AI into LMIC healthcare is a generational opportunity. We must move from theoretical promise to sustainable reality and architect a legal framework that is as innovative as the technology it seeks to govern. Only then can we ensure that AI serves as a powerful instrument of social equity rather than furthers the disparity.
At Esquare Legal, our Brazil Healthtech practice is directly engaged in this space — advising international technology companies navigating SUS procurement, LGPD health data compliance, and consortium structuring for public health digital transformation. Learn more about our Brazil Healthtech practice.
Hassan Raza is a Senior Associate at Esquare Legal, leading the firm’s trade finance, PayFi, and Brazil Healthtech practice. Contact us to discuss AI governance and health technology regulatory frameworks.