Happened in AI

Research watchMedicine

Google's AMIE chatbot interviewed 98 urgent-care patients under supervision in a Lancet feasibility study

A peer-reviewed Lancet study found Google's AMIE diagnostic chatbot could take pre-visit histories from 98 patients without a safety stop, but it was a small single-site study with no control group and did not measure patient outcomes.

Evidence: 5 sources · 2 official records · 1 independent report · 2 research papers · 4 open questions

By Happened in AI Editors

Published
Read
2 min

What happened

  • On 8 October, The Lancet published a prospective feasibility study of AMIE, a conversational diagnostic AI from Google, used before urgent primary care visits at Beth Israel Deaconess Medical Center in Boston.
  • Patients were enrolled between April and November 2025; 98 completed both the AMIE text conversation and their appointment with a doctor.
  • Supervising physicians watched each conversation in real time and did not need to stop any of them under the predefined safety criteria.
  • AMIE's list of possible diagnoses included the final diagnosis in about 90% of cases.

Why it matters

This is one of the first published tests of a patient-facing diagnostic chatbot with real patients rather than actors or written cases. It shows such a system can be run safely under close supervision, which is a first step, not proof that it improves care.

What is confirmed and what is claimed

Confirmed

  • The study was peer reviewed and published in The Lancet on 8 October, with an accompanying Lancet comment (Lancet; also reported by Medical Xpress and Unite.AI). 124
  • It was a single-center, single-arm study with no comparison group, registered as NCT06911398 (Lancet abstract; ClinicalTrials.gov). 125
  • No conversation had to be halted; supervisors noted one hallucination and added clinical information in five conversations (BIDMC and ARISE release; consistent across independent reports). 3
  • Doctors' management plans were rated better than AMIE's on practicality and cost-effectiveness (Lancet abstract; Medical Xpress). 124

Claimed, not independently verified

  • Google Research attributes AMIE's weaker management plans to its lack of access to health records, physical examination, and images or other non-text input.
  • Clinical evaluators rated AMIE's conversations favorably on most quality criteria; these ratings come from the study team's own assessment framework.

AMIE (Articulate Medical Intelligence Explorer) has been tested before in simulated consultations with trained actors, where Google reported strong results. The Lancet paper moves it one step closer to practice: real adults with urgent primary care problems chatted with it by text before seeing their own doctor, while a physician watched and could stop the conversation.

Context

In our evidence ladder, this is a peer-reviewed result, but of a particular kind. A feasibility study asks whether something can be done safely and acceptably, not whether it works better than the alternative. There was one site, no control group, and a supervising doctor in every conversation. AMIE also had no access to medical records, could not examine patients, and worked only with text. An earlier version of the work appeared as a preprint, and the published paper is the version to rely on; see our note on preprints and peer review.

AMIE is a research system. It is not a product cleared by the FDA or any other regulator.

What to watch

  • Whether the team runs a randomized or controlled study that measures what happens to patients.
  • Tests with less supervision, other languages, and more varied patient groups.
  • How regulators treat patient-facing diagnostic chatbots. The FDA's discussion paper on generative AI devices flagged patient-facing tools as potentially higher risk.

What we don’t know yet

  • Whether using AMIE changes diagnoses, treatment, or patient outcomes; the study was not designed to measure this.
  • How it would perform without a physician supervising every conversation, or in other clinics and patient groups.
  • Reported diagnostic figures beyond the 90% list-inclusion rate differ between outlets (for example, how often the top-ranked guess was correct), so we have left them out.
  • Some reports give 100 participants, the planned size in the earlier preprint; the published paper's completer count is 98.

Evidence and sources

  1. Peer-reviewed study Conversational diagnostic artificial intelligence in ambulatory primary care: a prospective feasibility study The Lancet ·
  2. Peer-reviewed study Conversational diagnostic AI in primary care: what happens after it speaks? The Lancet · Accompanying comment.
  3. Primary announcement ARISE, BIDMC and Google AMIE publish prospective clinical AI feasibility study in The Lancet ARISE
  4. Independent reporting First real-world study of safety and quality of patient-facing AI in primary care Medical Xpress
  5. Document or data AMIE's Clinical Conversational Abilities in an Urgent Care Setting (NCT06911398) ClinicalTrials.gov