AI in Healthcare: Sandboxes, Black Boxes, and Where Regulation Really Stands
A year ago I wrote that healthcare AI was moving faster than the rules meant to govern it, and that regulators were rapidly closing the gaps. The second half of that has turned out to be only partly true. In 2026 the picture is less a story of closing gaps than of diverging choices: US federal regulators have loosened some oversight, US states are tightening others, the EU has pushed back its deadlines, the UK is designing a staged licensing model, and in Utah an AI is now legally renewing prescriptions on its own.
How AI Operates Outside Traditional Regulation
Some of this is still loophole-driven, but more of it is now deliberate policy:
• Off-label use and clinical judgment: Physicians can use approved AI tools in ways regulators did not clear. Legal responsibility shifts to the clinician rather than the developer.
• Software that is not a "device": On January 6, 2026, the FDA revised its Clinical Decision Support guidance, extending enforcement discretion to tools that give a single clinically appropriate recommendation, and widened its general-wellness policy for non-invasive wearables. Both let more AI products reach the market without premarket review.
• In-house hospital tools: AI built and deployed inside a single health system, rather than sold as a product, can sit outside normal premarket pathways.
• Consumer health AI: In January 2026, OpenAI launched ChatGPT Health and Anthropic launched Claude for Healthcare, letting users connect medical records and wellness data to a chatbot. Consumer versions generally fall outside HIPAA, so protection comes from company policy, FTC rules and state privacy law rather than health-privacy law.
• State sandboxes: Utah’s AI regulatory sandbox now lets Doctronic’s autonomous system renew prescriptions for roughly 190 chronic-condition medications. The 12-month pilot began in January 2026, with physician review of the first 250 cases before the AI decides alone. Critics have questioned whether it needed FDA authorization and who is liable if it errs.
• Therapy chatbots, now restricted: Illinois and Nevada banned AI therapy in 2025. In 2026 at least five more states followed, including Maine, Tennessee and Vermont, and several others now bar chatbots from presenting themselves as mental health professionals.
Current Regulatory Responses
• FDA (United States): Predetermined Change Control Plans (final guidance, December 2024) let AI devices update along a pre-approved path. The agency has now authorized more than 1,350 AI-enabled devices. On August 18, 2026 it released a discussion paper on regulating generative-AI-enabled devices, covering foundation models and agentic AI; comments close October 19, 2026. The January 2026 guidance, by contrast, moved toward a lighter touch.
• European Union: The AI Act still classifies most healthcare AI as high-risk, but the Digital Omnibus (Regulation 2026/1744, in force July 27, 2026) delayed the deadlines. High-risk AI embedded in regulated products, including medical devices, now applies from August 2, 2028, and stand-alone high-risk systems from December 2, 2027. Transparency duties under Article 50 have applied since August 2, 2026, and the deadline for member states to open sandboxes moved to August 2, 2027.
• United Kingdom: The MHRA’s AI Airlock finished Phase 2 in May 2026 (seven innovators) and has secured £1.2 million a year for 2026–2029. On September 10, 2026 the National Commission into the Regulation of AI in Healthcare published 44 recommendations, including staged authorization (an "L-plates" model) and recording AI version information in the patient record.
• US states versus Washington: A December 11, 2025 executive order directed federal action against state AI laws it considers burdensome, but an order cannot itself preempt state law, and it carves out child safety. States kept legislating: by mid-2026, 84 new AI laws had been enacted in 27 states, including limits on AI-only insurance denials in seven states and disclosure rules for AI in patient care in Texas.
• Singapore: IMDA released the world’s first Model AI Governance Framework for Agentic AI in January 2026 (voluntary), and the HSA is developing a sandbox exempting certain AI software built by public healthcare institutions from standard registration.
Risks of Circumvention
• The "black box" problem: If an AI cannot explain its recommendation, it is hard for a doctor to obtain true informed consent. UK Airlock testing rated explainability a moderate regulatory gap and concluded requirements should vary by use.
• Liability gaps: Still unresolved. The Utah contract has been described as unclear on who is responsible if an AI prescribing error injures a patient.
• Algorithmic bias: Models trained on non-representative data can produce discriminatory outcomes that are missed when tools skip standard clinical validation.
• Privacy outside HIPAA: Records uploaded to consumer AI tools may lose health-privacy protections.
• Foundation-model dependency: The UK Commission warned that products built on a few externally owned models create a sovereignty and continuity risk.
When consent is generally advisable
• Clinical trials and sandboxes: Supervised by ethics boards or regulators with defined safety monitoring (as in Utah’s physician-review phase).
• Explainable outputs: A qualified human can interpret and act on the reasoning behind the recommendation.
• Human oversight: A physician remains in the loop and checks the output against your circumstances.
When consent is risky
• Opaque logic: If no one can explain it, valid informed consent is legally questionable.
• Automation bias: Doctors accepting AI output without questioning its limits.
• Unclear accountability: No one can say whether the developer, hospital or doctor answers for failure.
Safeguards to check before agreeing
• Disclosure: Ask whether the tool is FDA-cleared or authorized, and what its labeling says about limits. Some states, such as Texas, now require providers to tell patients when AI is involved in their care.
• Right to refuse: Most ethical frameworks, including WHO guidance, hold that essential care should not be denied if you decline AI-enhanced care.
• Validation for people like you: Ask whether the tool was tested on your demographic.
• Data protection: Ask where your data goes and whether HIPAA or equivalent law covers it.
Sandbox vs. Black Box
The two terms still describe different things: one is a regulatory method for safe testing, the other a technical problem with transparency.
The regulatory sandbox
A controlled environment in which developers test innovative AI under regulator supervision, with limited waivers or guidance. Examples include the UK’s AI Airlock, Singapore’s planned HSA sandbox, EU sandboxes due by August 2027, and Utah’s state program. A correction to my earlier version: the FDA does not run a sandbox of this kind; the US examples are at state level.
The black box
A model whose internal reasoning is too complex for even its designers to explain. Deep learning can be very accurate at tasks like spotting cancer on imaging yet unable to "show its work," which complicates consent and accountability. Generative AI has intensified the problem, which is partly why the FDA is now seeking input on foundation models and agentic systems.
How Regions Compare in 2026
Global strategies are diverging on how much they weigh innovation speed against transparency and risk control.
• Sandbox-led: United Kingdom. AI Airlock is funded through 2029, and the Commission’s staged-authorization proposal would build the sandbox idea into the licensing system itself. The MHRA says it intends to respond and begin operationalizing an updated framework, with implementation in 2027 where legislation is required.
• Sandbox-led and soft-law: Singapore. Voluntary frameworks, testing toolkits and targeted sandboxes rather than a statute-based risk classification. The UAE also operates a healthcare AI sandbox, with full local licensing still required for commercial rollout.
• Rules-led: European Union. Still the strictest on paper, with transparency, human oversight and documentation required for high-risk AI, but enforcement for medical-device AI is now a year later than first planned.
• Hybrid and fragmented: United States. Federal policy is lighter-touch on low-risk tools and uses PCCPs for adaptive AI, while states fill the gap with chatbot, disclosure and insurance rules. The federal government and states are in an open dispute over who sets the rules.
Verdict for late 2026: If you want to see where AI most actively tests the edges of traditional norms, look at the UK, Singapore and Utah. If you want to see where opaque AI is most strictly challenged, the EU remains the strictest on paper, but its medical-device deadline is now August 2028, so for the next two years the practical protections sit mostly with existing device law, state rules and your own questions.
Personal Note / Author’s Statement
This is general information, not medical or legal advice. Regulation here is changing monthly; check current sources before relying on any date above.
