AI Ethics 2.0: Why Frontier AI Demands a New Governance Agenda for Healthcare

This editorial appears in the July Issue of the American Journal of Bioethics

In February 2026, leaders from industry, government, policy, and academia gathered at New York University for a Summit on Building Governance Infrastructure for Frontier AI. The aim was ambitious and pressing: to develop governance principles for frontier AI systems before the technology outpaces the institutions responsible for overseeing it.

The stakes are high. Frontier AIs are agentic systems capable of interacting with and reshaping the world in countless ways. They can plan multi-step tasks, use external tools, remember across interactions, and operate with growing independence. Promising to deliver beneficial advancements, frontier AIs will soon be embedded in our vehicles, financial systems, schools, and, most relevant here, our hospitals.

Traditional AI systems are narrower in scope: a diagnostic algorithm, a scheduling optimizer, a billing classifier. Frontier AI systems, particularly agentic ones, represent something categorically different. They raise at least four distinct governance issues that challenge our current ethical and regulatory frameworks.

Dynamism: The risks of a frontier AI system are not static. These systems update, drift, and develop emergent behaviors. A system deemed low-risk at the point of deployment does not remain low-risk by virtue of that initial classification alone.

Autonomy: As AI systems carry out longer chains of actions with less human oversight, the potential for unintended consequences grows, partly because the system may begin operating well beyond its original scope.

Interaction: When multiple AI systems interact, coordinating tasks, sharing data, triggering each other’s actions, they can produce systemic risks that no single system would generate on its own. Governance must therefore address the ecosystem as a whole, not just individual tools in isolation.

Context-dependence: Risk emerges from the interaction between a system’s capabilities and its deployment context, user population, level of autonomy, and the reversibility of its decisions. Identical systems deployed in different settings may call for fundamentally different governance approaches.

These are not hypothetical concerns. Agentic AI systems are already at work across healthcare, and their role goes well beyond transcribing clinical notes. At Oxford University Hospitals, a multi-agent AI system called TrustedMDT is being piloted in cancer tumor boards. One agent summarizes patient records across radiology, pathology, and biomarker tests; a second determines cancer staging using international standards; and a third drafts guideline-compliant treatment plans for review by the multidisciplinary team. Epic Systems, which serves approximately thirty-eight percent of U.S. inpatient facilities and holds 325 million patient records, has deployed multiple AI agents: Emmie for patient engagement, Art for provider communications, and Penny for revenue cycle management. Hippocratic AI’s voice agents autonomously call patients to schedule screenings and tests and to handle follow-up. To date, these agents have logged over 115 million clinical patient interactions across more than fifty health systems including Cleveland Clinic, Northwestern Medicine, and Ochsner Health. A study published in NEJM AI reports that forty-three percent of surveyed health systems are already piloting agentic AI, although only three percent have moved agents in live clinical workflows. Sixty-one percent of health care technology executives report that they are building or implementing agentic AI initiatives or have secured budgets to do so, and eighty-five percent plan to increase investment over the next two to three years.

Bioethics has made important contributions to AI ethics, tackling algorithmic bias, fairness metrics, explainability, and data privacy. But frontier AI systems pose new problems that demand new structures. When an agentic system coordinates a multi-step clinical workflow with minimal oversight, the question is not just whether the algorithm is biased. The questions are: How do we govern systems whose risk profiles change over time? Who bears responsibility when harms accumulate slowly across thousands of interactions? Even today, some clinicians are beginning to defer to algorithmic recommendations. A systematic review found that in six percent of cases, clinicians overrode their own correct decisions in favor of flawed advice from decision support systems. A randomized crossover study showed that clinicians at every level of expertise were vulnerable to automation bias. Frontier AI systems, which are more persuasive, more autonomous, more deeply woven into clinical workflows, threaten to accelerate this erosion of clinical judgment considerably.

What is needed is a new kind of governance infrastructure, what we are calling AI Ethics 2.0. The first wave of AI ethics focused on properties of individual models: Is this algorithm fair? Is it transparent? Does it violate privacy? Those questions still matter. But governing frontier AI requires a shift from model-level analysis to institutional governance, and from static risk classification to dynamic, continuous oversight.

The NYU Summit’s working groups started from a blunt premise: the absence of comprehensive regulation does not excuse the absence of governance. For hospitals, this means the work cannot wait for legislation.

Much more will need to be said, but several steps are already clear. First, hospitals and health systems need to establish dedicated AI governance structures. These should not be add-ons to existing IT committees, but standing bodies with ethical, clinical, and technical expertise, and with real authority to approve, condition, suspend, or withdraw AI deployments. Currently, only eighteen percent of health systems have an enterprise-wide AI governance strategy.

Second, hospitals and health systems should implement ongoing monitoring and re-review processes. AI systems that update continuously cannot be governed by one-time approval. Health systems should build incident response plans before deployment, not after harm occurs, and should create channels for clinicians to report AI failures without professional penalty.

Third, professional associations including medical boards, nursing boards, and specialty societies should establish AI competency requirements for their members. Allowing professionals to use AI without demonstrated understanding of its limitations puts patients at risk.

Fourth, procurement should be treated as an ethical act. Health systems have compressed average AI buying cycles from 8.0 months to 6.6 months. Faster procurement means less time for governance review, less time for clinical validation, and less time for the kind of ethical scrutiny that frontier AI demands. At a minimum, procurement contracts should include transparency provisions, audit rights, incident reporting obligations, and clear allocation of liability between vendor and deployer.

AI governance is already happening in hospitals whether or not anyone calls it that. Every procurement decision, every pilot approval, every choice to let clinicians use an AI tool without formal oversight is itself a governance decision, just an unexamined one. The question is not whether hospitals will govern AI, but whether they will do so deliberately, with structures built for the challenges frontier AI presents, or whether they will continue making these decisions by default and reckon with the consequences later. Bioethics should help ensure it happens ethically, with governance structures that can adapt as the technology, the evidence, and the stakes continue to evolve.

S. Matthew Liao, PhD, and Jennifer Blumenthal-Barby, PhD