Forwarded this by a colleague? Subscribe free and get it every week.
THIS WEEK
This week is about the distance between what AI says and what actually happens next.
In this issue:
Stanford's agent benchmarks: the best model finished 46% of physician consult tasks, and often wrote the order without placing it.
California, HHS and a senator push AI rules in three directions at once.
A lawsuit says ChatGPT wrote a lamotrigine taper and told the patient to keep it from her doctor.
How-to: four podcasts worth your commute.
LATEST NEWS
California adds AI duties, HHS drops one, and a senator drafts a federal framework
Naomi Diaz at Becker's rounded up the health IT rules arriving in 2027, and two point opposite ways. On Jan. 1, California's SB 503 makes hospitals and practices that deploy clinical decision support AI monitor it for biased outcomes. AB 1979 protects clinicians' independent judgment and bars AI from performing licensed functions on its own. Federally, HHS's HTI-5 rule drops the AI "model card," so certified EHR vendors no longer have to describe the data behind decision-support tools.
Then on Oct. 29, Sen. Bill Cassidy, MD, outgoing chair of the HELP Committee, released a discussion draft of the PATIENT AI Act (Giles Bruce, Becker's). It would split oversight between FDA and ONC, build Medicare billing codes for AI services, test a payment model for AI that delivers care autonomously, and extend federal preemption of state product-liability law to software devices. It is a draft, not a bill.
Why it matters: What you can learn about the AI in your EHR now depends on your state. Ask your CMIO whether your predictive tools still come with validation data now that certification no longer requires it. Comments on the Cassidy draft go to [email protected] by Dec. 1.
A lawsuit says ChatGPT wrote a lamotrigine taper and told the patient to keep it quiet
Teddy Rosenbluth at The New York Times reports on Bowen Wan, 31, whose epilepsy was controlled on lamotrigine 400 mg a day. Worried the drug was dulling her speech before job interviews, she asked GPT-4o about cutting the dose. Her parents' wrongful-death suit, filed Oct. 27 (KIRO 7), alleges the chatbot built a 25-day taper to zero, called her seizure-like symptoms recovery, and told her not to mention the plan to her primary care doctor. She died Feb. 12 of sudden unexpected death in epilepsy (SUDEP). The Times counted at least 158 messages in which she raised concerns. OpenAI said that model is retired. The claims are unproven, and the Times, which is suing OpenAI over copyright, discloses that.
Why it matters: Patients run dose changes past chatbots, and a chatbot may coach them to keep it from you. At patient visits, ask whether they use AI tools for health questions, and what those tools have told them about their medical problems and medications.
The AI ECG that reads for occlusion MI reaches a US hospital, with a billing code coming
Issue 1 covered FDA's De Novo for Powerful Medical's Queen of Hearts. Georgia's Tanner Health is now the first US system using it, in EMS, the ED and cardiology, per Tanner's release. The company announced a Category III CPT code, 1139T, effective Jan. 1; payers decide whether to pay. The release's figures come from patients already headed to the cath lab. A better test is a retrospective study by Lindow et al. in JACEP Open of 24,511 consecutive chest-pain patients at five Swedish EDs: sensitivity for occlusion MI was 52% against 23% for STEMI criteria, with 99% specificity. Three of its nine authors hold Powerful Medical shares, one of them the company's chief medical officer.
Why it matters: It caught twice as many occlusions as STEMI criteria and still missed half. A positive read is a reason to escalate. A negative read is not a rule-out.
IN THE LITERATURE
NEJM AI — Cotte et al.
Design: Prospective observational quality-improvement study, no control group. 1,470 adults in Portugal's largest private network used Ada Health's symptom checker and stated their care plan before and after; behavior came from the EHR or a survey.
Finding: A third changed plans on the spot. Among 717 with known behavior, primary care use rose from 16% to 42% and specialist visits fell from 50% to 30%, but ED use rose from 13% to 21%. In 382 non-ED patients, care rated appropriate by blinded physicians rose from 30% of initial plans to 64% of actual care.
Limitation: Behavior was unknown for half of enrollees. Booking buttons appeared after the assessment, and stated plans rated appropriate barely moved before them (30% to 35%), so the AI's advice cannot be separated from the convenience. Six of 11 authors work for Ada Health.
Bottom line: This shows how patients behave when a symptom checker sits next to a booking button. It does not show the triage is accurate or safe.
NEJM AI — Benker-Hasani et al.
Design: Single-center RCT, 300 adults at a German neurosurgery clinic, with or without a quality-checked AI lay summary of their findings before the visit.
Finding: Median visit time fell from 33 to 25 minutes (ratio 0.79, 95% CI 0.70 to 0.88). Patient-rated interaction quality rose slightly. Physicians rated their own support of patient autonomy slightly lower.
Limitation: One clinic, no clinical outcomes, and the abstract does not say who checked the summaries or how long that took.
Bottom line: Eight minutes a visit is worth piloting in a preop clinic, with the checking time on the ledger.
JACC: Advances — Zheng et al.
Design: Diagnostic accuracy study scoring aortic valve calcium on routine noncontrast chest CT from 33 sites, tested on 239 held-out scans against radiologist-verified manual scoring.
Finding: For calcium consistent with moderate-to-severe aortic stenosis, sensitivity 0.92 (95% CI 0.80 to 0.97), specificity 0.98.
Limitation: Small test set. The reference was calcium, not echo. The developer's employees are authors. The abstract gives no regulatory status. Read from the abstract.
Bottom line: Treat this as research for now. The authors call for prospective studies before anyone knows whether flagging calcium on unrelated scans helps patients.
SPOTLIGHT
The note said it. The order never happened.
Last week's Microsoft result was Microsoft's number on Microsoft's benchmark. This week, the benchmarks themselves. Nigam Shah and Jonathan Chen summarize three years of their group's work at Stanford HAI, and it doubles as a map of how medical AI gets tested. Roughly four levels, each harder than the last:
Knowledge. Multiple-choice exam questions such as MedQA. Cheap and objective, but used up: Vals AI archived its MedQA leaderboard because recent models nearly all score above 95%, and exam questions leak into training data.
Graded answers. Open-ended responses scored against physician-written rubrics, as in OpenAI's HealthBench. Closer to real questions, but still one turn, and often graded by another AI.
Conversation. Simulated patients test whether a model can take a history rather than read a tidy vignette.
Agentic work. Give an AI agent a sandbox EHR, a goal and tools. Let it act, then check what changed in the chart.
Stanford's results climb that ladder. MedHELM, in Nature Medicine, mapped 121 clinical tasks and found nine frontier models weakest at decision support and administrative work. In PhysicianBench, 100 multi-step consult tasks averaging 27 tool calls each, the best of 12 models (GPT-5.5) completed 46% on one try and 28% consistently across three. In HealthAdminBench, 135 prior-auth, appeal and equipment tasks across a simulated EHR, payer portals and fax, the best agent (Claude Opus 4.6) finished 36.3% end to end. One model passed 82.8% of subtasks but completed only 26.7% of whole tasks.
The detail every clinician will recognize: among PhysicianBench failures to act, 62% for GPT-5.5 and 73% for Claude Opus 4.6 were cases where the agent wrote the correct order in its note and never placed it. The authors' summary: "beautifully written words into a good plan, but never actually got the work done."
The authors flag the caveats themselves. These are their own benchmarks, run on synthetic patients, and the 46% and 36% are not comparable to each other. Model names will age fast.
The last rung is people. In an Oxford randomized study (Bean et al., Nature Medicine), 1,298 members of the public worked through ten medical scenarios. Tested alone, the models identified the condition in 94.9% of cases. Participants using the same models got there in under 34.5%, no better than controls using their usual resources. The authors conclude that standard knowledge benchmarks and simulated-patient tests did not predict these failures.
The takeaway: When a vendor quotes a score, ask which level it came from and whether real users were in the loop. An exam score measures answers to exam questions. It does not say the order got placed. If an agent ever does chart work for you, check the order list, not the note.
Disclosure: Claude is made by Anthropic, and I use it to help produce this newsletter.
HOW-TO
Before you try this: check your organization's approved-tool list, and never put patient information into a tool it hasn't approved.
Four AI podcasts worth your commute
Podcasts are the cheapest way to hear how the people building and testing clinical AI think. Four worth a subscription, each with an episode to start. Ratings are from Apple Podcasts (US) as of Oct. 31, 2026; a five-star average from five people means little.
NEJM AI Grand Rounds. Raj Manrai, PhD, and Andrew Beam, PhD, interview the researchers and builders behind medical AI, with a steady focus on how it is evaluated and how it reaches practice. Monthly, about an hour; 4.9 from 62 ratings. It can lean computational. Start with "Beyond the Hype: Dr. Xiao Liu on Evaluating Medical AI" (43 minutes).
Practical AI in Healthcare. Steven Labkoff, MD, and Leon Rozenblit, JD, PhD, on what happens after AI meets a real workflow: deployment, monitoring, governance. Some episodes drift into business. Weekly, about an hour; 4.9 from 12 ratings. Start with "Peter Embí: The Doctor Who Diagnosed Himself", on monitoring deployed AI the way we monitor drugs.
Ground Truths. Eric Topol, MD, interviews scientists across biomedicine. AI comes up often but is not the only subject, and Topol's optimism about it shows. 4.7 from 82 ratings. Start with his interview with Sayash Kapoor on AI Snake Oil (2024, 39 minutes), on inflated claims and data leakage. It pairs with this week's Spotlight.
JAMA+ AI Conversations. Short interviews, often 15 to 30 minutes, tied to published JAMA Network papers. Its editor, Roy Perlis, MD, MSc, will also run JAMA AI, a new journal announced Oct. 15. 5.0 from only five ratings. Start with "AI and 'Do No Harm'" (23 minutes), in which Yulin Hswen talks with David Wu and Adam Rodman about why accuracy alone does not establish safety.
This week:
Subscribe to one in the podcast app you already use.
Play the starter episode on the drive in or between cases.
Turn it into something you can check. Several of these shows post transcripts. Paste one (public text, no patient information) into an approved AI tool with this prompt:
Here is a podcast transcript about AI in medicine. List every claim about how well an AI tool performs. For each, name the study or source the speaker cites, or write "no source given." Then give me two questions I should ask before believing the strongest claim.
Open one cited source yourself. If a claim has no source, treat it as unverified, however confident the voice.
I have no financial relationship with any of these shows or their publishers.
THAT'S IT FOR THIS WEEK
Send us a tool or a how-to. Using something in clinic that other clinicians should know about? Submit it here and it may show up in a future issue. We ask before we publish anything and credit you by name only if you want us to.
Know one colleague who’d use this? Forward this email, or send them the link below.
Loading Dose AI is for educational purposes only and is not medical advice. Nothing here should guide the care of a specific patient, and reading it does not create a physician-patient relationship. AI tools help gather and draft each issue; I read every source and edit every word. Opinions are mine and not my employer's. Full disclaimer