Who pays for AI oversight?
This week, I attended a convening at Duke University in which I got to hear from a variety of health care providers about how they're using AI.
Most interesting for this audience: representatives of critical access hospitals and Federal Qualified Health Centers said in the bill vs. buy debate they are all in on buy. Obviously some startups have built their business around going after this specific market, as I've written about before. But the market still seems underserved.
The event was hosted by Duke University's Health AI Partnership in Durham, North Carolina. Health AI Partnership is an effort to help lesser resourced care settings use AI safely. Though it was sponsored by Oracle, Vega Health, and the MacArthur Foundation, I attended at my own cost.
A variety of people attended: chief innovation officers, policy experts, health systems, electronic health record employees, and health system executives. What stuck out to me during their discussions was how much help is still needed in lesser resource settings for building the infrastructure needed to monitor AI. These health providers are no longer just concerned about evaluating AI products, but tracking their performance post launch.
Larger health systems have the money to deploy this kind of supervisory system. Critical access hospitals and publicly funded clinics don't. And this infrastructure is costly. It requires heavy training, technical staff, and the ability to build a monitoring apparatus.
That raises a question: should frontier model developers and other AI companies be responsible for building and implementing a governance structure for these products?
The consensus in the room seemed to be yes.
Of course, as we saw last week with Utah's announcement, there are companies that are trying to fill this gap. So, I'm curious what our audience thinks. How should an AI governance system work in health care and who pays for it?
Now onto the news…
—Ruth
Four Questions with Lisa Shah
Christina Farr asked Lisa Shah about the employers now pulling back on GLP-1 coverage — and what they should have done instead. She joins us on Oct. 28 for Type 2 Diabetes & Cardiometabolic Care: Wearables, AI & GLP-1s.
Lisa Shah, M.D. is executive vice president and chief medical officer of Twin Health, the digital twin care platform for metabolic disease. She previously served as CMO of Advantia Health, chief innovation officer of Sound Physicians, and SVP of clinical operations and network at Evolent Health.
Lisa Shah: Yes, this is definitely happening, and we're hearing a lot about it. It's a difficult market trend, and it shows that when we make quick decisions about drug coverage without all of the information (cost, utilization, long-term spend and return on investment), the negative impact can really land on the person.
A lot of companies moved quickly to make this coverage decision instead of slowly testing the concept. Then they realize very quickly that their costs went up dramatically and the ROI on the weight loss wasn't there, especially with turnover in their organizations. Now they're forced into another extreme response, which is removing coverage, and many are removing it completely. The consequences of that are significant, because now you have a rebound weight gain problem with no off-ramp.
My hot take is that employers shouldn't be covering these drugs without a lifestyle benefit alongside them. If they did, they probably wouldn't be dropping coverage as much. They'd see more ROI, because they'd be able to get people off the drugs without rebound weight gain.
Read the other three questions →
Upcoming Sessions
Live conversations with the people shaping healthcare.
Is It Finally the Moment for Alternative Plan Design?
Employers are under growing pressure to control healthcare costs while improving access, outcomes, and the member experience. That has renewed interest in
Type 2 Diabetes & Cardiometabolic Care: Wearables, AI & GLP-1s
The rise of wearables, GLP-1s, and AI is reshaping treatment for Type 2 diabetes, weight loss, and metabolic disease. We have
Verifying AI in Medicine: Is Healthcare AI Actually Ready for Deployment?
Is AI ready for real-world clinical care? Dr. Zak Kohane and Protege’s Engy Ziedan debate what it takes to safely trust AI in diagnosis and treatment, from benchmark performance and clinical evidence to whether we’re holding AI to a higher standard than humans.
Why Are Hospitals Hiring Chief AI Officers?
As AI moves from pilots into day-to-day healthcare operations, health systems are having to decide who should actually own the
Where are AI Agents Actually Working in Healthcare, and Why?
While the healthcare industry remains locked in a high-level debate over the theoretical potential of artificial intelligence, a select group of
Top Employers Share What Healthcare Startups Need to Know in 2026
As enterprise health purchasers face volatile medical cost trends and tightening operational margins, the standard vendor pitch of "improved clinical efficiency&
Peptide Therapy: Where Do We Go From Here?
Watch the full webinar Peptide therapies have moved from niche biohacking circles into mainstream medicine, reshaping conversations around weight management, metabolic health,
Not everyone can access the top 1% of physicians. Will AI change that?
Historically, top-tier medical expertise was limited by wealth and geography. As AI reshapes healthcare delivery, a vital question emerges: can it democratize intelligence to give everyone access to the caliber of care typically reserved for the top 1%?
Privacy AI and the Future of HIPAA
Watch the Full Webinar As AI reshapes healthcare and more patient data flows beyond traditional clinical settings, the rules governing privacy, access,