Assurance labs are back

On Monday, the Utah Department of Commerce will announce a roster of AI validation and oversight companies that can vet and oversee participants in its AI sandbox, according to two people with knowledge of the announcement.

Back in July, I reported that Utah was looking into the role independent third parties could play in verifying AI health products and providing oversight for its AI sandbox. In August, the state put out a paper that laid out its perspective on what kind of evidence companies in its AI sandbox need to generate to demonstrate both public health utility and safety. The paper also said AI sandbox applicants should include plans for third party validation of their internal findings. Now, the Utah Department of Commerce is hosting an event wherein it will announce several companies that sandbox applicants can engage to vet and monitor their technology—and provide data back to Utah— for the duration of their sandbox pilot.

The independent third parties will in some cases partner with health systems to validate AI technology that's used at the point of care. In the case where a sandbox product is part of a platform like Doctronic, let's say, the third party will monitor, validate, or offer other governance services directly within the platform.

Why it matters: Utah has emerged as one of the country's most important testing grounds for how AI in health care should be regulated. Anyone building technology in this space should be paying attention. The way that it develops its sandbox program has wide repercussions as both other states and federal regulators try to figure out how to oversee health AI products.

Are these assurance labs? Some of you may remember the stir around AI assurance labs that happened around 2024. The Coalition for Health AI, an organization originally comprising representatives from big-name health providers and tech companies including Mayo Clinic, Duke Health, Microsoft, Amazon, and Google, sought to create a framework for validating health AI products. The idea was that assurance labs, independent third-party evaluators, would vet health AI algorithms. It even had support from the Department of Health and Human Services under President Joe Biden.

However, the assurance lab concept fizzled before it even really got started as industry consensus started to coalesce around the idea that AI needed to be vetted locally, inside of the health systems where it was being deployed. Then, under President Donald Trump, HHS publicly rejected CHAI, suggesting that it was a cover for regulatory capture under big tech.

Well, Utah is bringing assurance labs back effectively. I would expect CHAI to make an appearance next week as Utah embraces its new independent third-party AI evaluator program.

— Ruth

And now onto the news…