Abhijith Varma Mudunuri

AI scribes are changing what a traditional clinic visit feels like. At UCSF Health and other major health systems, after a patient’s consent, these tools listen to the conversation between clinician and patient, then draft the clinical note automatically. Patients receive a visit that feels more personal, as clinicians no longer need to type, write, or dictate their own notes, and can instead give their full attention, answer questions, and focus on care.

Before any AI drafted note enters the medical record, the clinician reviews and edits it for accuracy. That review is an essential safeguard, but it raises a question that remains largely unanswered: how often do these tools make mistakes, and how harmful could those mistakes be if they slip through?

Abhijith Varma Mudunuri, an undergraduate student researcher in the Alaa Lab and Berkeley AI Research (BAIR), is working to close that gap. Alongside Edge Medicine Center Lead Ahmed Alaa, they are building a platform where physicians review AI generated notes against the actual visit audio and score every error for its potential to cause patient harm. The result is a rigorous, independent measure of how often AI scribes get it wrong and how much those errors matter.

For Abhijith, independent academic validation and benchmarking are crucial for earning public trust in ambient AI. “AI is becoming ubiquitous in healthcare, but most of these tools are evaluated internally by the companies that sell them,” he says. That leaves health systems, clinicians, and patients without an independent view of how safe these tools actually are, and traditional evaluation cannot fill the gap. Randomized trials often take years and are designed for fixed interventions like drugs, not AI models that change with every patient visit, and they produce population level estimates that say little about a specific patient or visit. This work offers a faster, more precise alternative: measuring performance continuously, error by error, as the technology evolves. “As care shifts from the clinic into the home, ambient AI assistants will take on an increasing share of the clinical work,” he says.

“I’m proud that my work is building the ground truth evaluation layer that quantifies how much we can trust this vital technology.”

Abhijith’s work shows what becomes possible when rigorous evaluation meets clinical AI. It is a glimpse of the future Edge Medicine is building toward, where the technology supporting care is held to the same standard as care itself, and where patients and clinicians can trust the tools working alongside them.