What AI can and cannot do in healthcare right now
We build a healthcare product with AI in it, which is exactly why it is worth writing down what the research says, including the parts that are inconvenient for us.
We build a healthcare product with AI in it, so this article has an obvious conflict of interest. That is exactly why it is worth writing down plainly what the evidence currently says, including the parts that are inconvenient for us.
Where the evidence is least flattering
The most useful study to know about was published in Nature Medicine in 2024 by Hager and colleagues. It tested large language models against clinicians on real diagnostic decision-making rather than on exam questions, which is a much harder and more honest test.
The result was not close. The authors found that "current LLMs perform significantly worse than clinicians on aggregate across all diseases", with gaps of roughly 16 to 25 percentage points. Physicians scored between about 87 and 92 percent; the best model managed around 68 percent.
Their conclusion is worth quoting in full, because it is unusually blunt for a research paper. Without extensive physician supervision, they wrote, these models "would reduce the quality of care that patients receive and are currently unfit for the task of autonomous clinical decision-making".
Two failure modes stood out. The models tended to reach a diagnosis before gathering all the relevant facts, and they were confident while doing it. That combination is worse than being wrong slowly, because confidence is what makes a wrong answer persuasive.
The word for the second problem is hallucination
A 2025 review in Healthcare describes it as "confidently stated inaccuracies", and notes these "pose serious risks in a field where errors can have critical consequences".
The important part is the mechanism. These systems produce fluent, plausible text, and fluency is not a signal of accuracy. A fabricated drug dose reads exactly like a correct one. An invented citation looks like a real citation. Nothing in the output announces which is which, which is why "it sounded authoritative" is not evidence of anything.
Where it genuinely helps
The same review is positive about a different set of tasks. It finds that language models "effectively automate tasks like clinical note summarization, data extraction, and report generation, potentially reducing administrative burdens", and sees promise in medical education, as "virtual patients, personalized tutors, and tools for generating study materials". It describes potential in "diagnostic assistance, treatment recommendations, and medical knowledge retrieval".
Notice what those have in common. They are tasks where a person checks the output, and where being wrong is recoverable. Summarising a note that a clinician then reads is a different risk category from telling a patient what is wrong with them.
The review's own conclusion draws the line in one phrase: these systems should function as "decision support tools rather than autonomous decision makers", and safe implementation "depends on thorough validation and robust oversight".
What that means for anyone using a health chatbot
Three things follow from the evidence, and they apply to any such product, ours included.
Confidence is not accuracy. The tone of an answer carries no information about whether it is right. Treat a fluent explanation as a starting point for a conversation with a clinician, not as a substitute for one.
What the system can see determines what it can say. A model answering with no access to someone's chart, medications or history is reasoning about an average person. That is a different task from reasoning about your father, and the answers should not be trusted equally.
The question is who checks the output. This is the part worth asking of any health AI you encounter. Is a licensed clinician in the loop before an answer becomes a decision, or is the software the last step? The research is consistent that the second arrangement is not currently safe.
Where we stand
This evidence is the reason Dr.life is built the way it is. The AI answers questions and prepares the ground. It does not diagnose, prescribe, or decide on treatment, and it brings in a licensed Life Medical clinician when a question needs judgement. That is not caution for its own sake. It is what the published work says is required.
We would rather tell you what the research says about the limits of this technology than let the product's marketing outrun it.
Sources
- Evaluation and mitigation of the limitations of large language models in clinical decision-making. Hager P, Jungmann F, Holland R, et al. Nature Medicine, 2024. Used for the comparison against clinicians, the accuracy figures, the premature-diagnosis finding, and the conclusion on physician supervision.
- A Review of Large Language Models in Medical Education, Clinical Decision Support, and Healthcare Administration. Vrdoljak J, Boban Z, Vilović M, Kumrić M, Božić J. Healthcare (Basel), 2025. Used for the description of hallucination, the areas where these systems are useful, and the conclusion on oversight and decision support.
Every source above was read before it was cited. Where the evidence is uncertain, this article says so rather than rounding it into advice.