Undiagnosed No More: finding the patients the health record misses

Approved

Team Undiagnosed No More (UNOM)

  • Amir Asiaee, PhD, Assistant Professor, Department of Biostatistics and Biomedical Informatics, Vanderbilt University Medical Center (amir.asiaeetaheri@vumc.org): team lead, machine learning models, and All of Us analyses.

  • Kaveh Aryan, PhD, Postdoctoral Researcher, Department of Informatics, King's College London (kaveh.aryan@kcl.ac.uk): clinician and patient applications, and EHR integration.

The problem. Nine invisible illnesses are often missing from the health record even after a clinician names them, and a patient without the code is invisible to referrals, quality programs, trials, and research. In the NIH All of Us Research Program, about 1 in 9 adults who ever had Long COVID carry its code, and women's Long COVID is coded 0.79 times as often as men's. Among people in care for a diagnosis they report, 24% with ME/CFS, 39% with Lyme disease, and 41% with endometriosis have its code, against 85% and 87% for hypertension and type 2 diabetes.

What we cover. Nine invisible illnesses in five families: infection-associated chronic illness (Long COVID, ME/CFS, Lyme disease), autoimmune (lupus), chronic pain (fibromyalgia, migraine), women's health (endometriosis), and mental health (depression, anxiety). For every one of the nine, UNOM provides four things:

  • A coded share: how often people who say a clinician diagnosed them actually have that diagnosis code in their health record, measured in the NIH All of Us Research Program.

  • A predictive model: a machine learning model, built from health record data, that flags charts that look like those of people with the illness.

  • A clinician card: a summary inside the health record that shows a patient's estimate, the record entries behind it, and what to check next.

  • An Atlas page: maps and charts of who goes uncoded, by state, sex, age, and race and ethnicity.

Six of the nine also have questionnaires in the patient app.

How it works. U.S. government data trains the UNOM AI models, and the models serve four kinds of users. We first describe the data and the models, and then what each user does with them.

The data. Four U.S. government sources:

  • The NIH All of Us Research Program: health records linked to survey answers, and Fitbit data. We study it only inside its secure Researcher Workbench, and only privacy-checked aggregate results leave it.

  • The CDC: the Household Pulse Survey, the National Health Interview Survey, NHANES, and Lyme disease surveillance, for how common each illness is and where.

  • The U.S. Census Bureau: county population, to turn case counts into rates.

  • CMS: Medicare physician payments, to put a price on missed diagnoses.

The CDC, Census, and CMS data are public files.

The UNOM AI models. Nine machine learning models share one scoring engine.

  • Each model is trained outside the All of Us Workbench from its released aggregate counts, so no individual's data ever leaves it.

  • Eight models learn from patients' own reports that a clinician diagnosed them. They learn to find people whose diagnosis was never coded, and on all eight they rank those people higher than models trained on diagnosis codes. The Long COVID model learns from a later Long COVID code.

  • Given a patient's health record, the scoring engine returns the estimated chance of the pattern, how that compares with the average patient, a band (typical, elevated, or high), and the dated record entries behind it. The same engine runs in the clinician app, the EHR integration service, and the developer page, so the same chart gets the same result everywhere.

The four users. We built UNOM for clinicians, patients, public policy makers, and developers. Each has its own door on the website.

For clinicians (Clinician app). A primary care clinician sees a patient with fatigue, pain, or new symptoms after COVID-19 and no diagnosis that explains them.

  • The clinician opens UNOM from inside the electronic health record (EHR), with no new login. It launches through SMART on FHIR, the standard way to plug an app into an EHR, and runs today against the public SMART sandbox. On the website, the same app runs on synthetic charts.

  • UNOM reads the patient's chart and checks it against all nine models. "Needs review" lists the conditions whose pattern stands out.

  • For each condition, a card shows the band (typical, elevated, or high) and how this patient's estimate compares with the average patient's (for example, 4 times the average). It lists every dated entry behind the estimate (a diagnosis, a lab test, a prescription, a visit), with an arrow and a bar showing how much that entry raises or lowers the estimate. It shows the published diagnostic criteria to check, such as the 2024 National Academies definition of Long COVID, and a concrete next step, such as a post-COVID clinic referral, CDC's two-step Lyme test, or an ANA blood test for lupus.

  • If the patient has filled in the patient app, the answers update the estimate through published likelihood ratios.

  • A health system can also receive the same cards automatically through a CDS Hooks service (a synthetic-data prototype).

For patients (Patient app). A person with long-lasting symptoms wants to prepare for a visit and be taken seriously.

  • They open the patient app on a phone or computer, in English or Spanish. There is no account and no tracking, and the answers stay on the device. A caregiver can answer for someone else.

  • They answer validated questionnaires: the NIH RECOVER symptom questions, the DePaul questions on crashes after activity (post-exertional malaise), the PCFS daily-life scale, a symptom timeline, and self-checks they choose (Long COVID, ME/CFS, fibromyalgia, Lyme disease exposure, the PHQ-9 for depression, and the GAD-7 for anxiety).

  • The last page is a color-coded summary, one card per questionnaire, that says how the answers compare with the published criteria and what to discuss with a clinician. It is not a diagnosis.

  • They print a visit summary, or download it as a FHIR QuestionnaireResponse, a standard file clinic software can read. With Send to my clinician, they open the clinician view with their answers attached and see the estimate change. In "Try it yourself" mode, anyone can type their own answers and attach them to a patient's chart.

  • They can see which patterns in a health record are common in people diagnosed with each illness. Today the app runs our nine models on a record, on the device, and explains the patterns in plain words. Connecting a patient's own record is planned, through the patient-access connection the 21st Century Cures Act requires health systems to offer.

  • They can search ClinicalTrials.gov for studies recruiting now. Only the condition and the place are sent.

For public policy (Atlas). A health department, payer, or policy maker wants to know who is missed and what it costs.

  • They pick one of the nine illnesses.

  • They see the coded share by state, who goes uncoded by sex, age, and race and ethnicity, what else is in the records of people with the illness, and their Fitbit activity and sleep where available.

  • For Long COVID, a cost calculator estimates the clinician time per patient found by a two-step screen, and they can change every assumption.

For developers (Developer documentation). An EHR vendor, a health system's IT team, or a researcher wants to run or check our models.

  • They download the nine versioned model files, each with a model card (what it predicts and for whom) and a hash that confirms the file.

  • They run the scoring engine in their own code, or try it in the browser on any of 18 synthetic test patients, and compare the results with 180 expected outputs.

  • They connect an EHR through the documented SMART on FHIR launch, or by calling the CDS Hooks service.

The patient's voice changes the clinical picture. In the clinician app, a patient's answers update the estimate through published likelihood ratios. Maria's Long COVID is 6.1 times the average from her chart alone and 15.4 times with her answers. With "Try it yourself" and Send to my clinician, anyone can type their own answers and watch them move a patient's estimate.

Lived experience. We learned from the ME/CFS patient advocates who spoke in the TOPx Invisible Illness sessions and from a Lyme patient survey shared with sprint teams: patients are disbelieved even with test results, many are too sick to argue their case, visits are short, and patients rebuild their history before every visit and want to approve what is shared. The clinician flag with its evidence, the visit summary the patient approves, the caregiver mode, and the sources on every card answer those points.

Evidence. Four papers (preprints submitted for journal publication) document the coding gaps, the models, and the system. They are linked from the About page.

Solution Overview (MVP Demonstration)

Data & AI Solution

Live demo, prototype, or working product

Everything runs now, with synthetic patients only and no login.

Access instructions (if needed)

No account or login is needed, and every patient is synthetic. Start at https://undiagnosednomore.org/challenge/: it follows the four criteria in this form and links each point to the part of the demo that shows it.

  1. For clinicians: open https://undiagnosednomore.org/clinician/ and pick Maria, Elena, Sofia, or Dana. "Needs review" lists the flagged conditions (Maria: Long COVID). Hover a bar or an underlined number for its explanation. The "Launch via SMART on FHIR" link opens the public SMART sandbox: choose any of the 18 test patients and the app reads that chart over FHIR.

  2. For patients: open https://undiagnosednomore.org/patient/. It opens on a synthetic patient's answers. Use "Try it yourself" for a blank form, then Send to my clinician from the summary to see your answers move a patient's estimate. Answers stay in your browser. Switch to Spanish with "Español" at the top.

  3. For public policy: open https://undiagnosednomore.org/atlas/ and pick a condition.

  4. For developers: https://undiagnosednomore.org/developers/ lists the model files, runs the scorer in the browser on any test patient, and sends example requests to the CDS Hooks service.

  5. To see everything in a few minutes: https://undiagnosednomore.org/walkthrough/ has the 4-minute walkthrough video and the slide deck.

  6. Works in current Chrome, Edge, Firefox, and Safari, on desktop and phone.

Results for the American People

Millions of Americans live with an illness their record does not show. In the NIH All of Us Research Program, about 1 in 9 adults who ever had Long COVID carry its code, and women's is coded 0.79 times as often as men's. Of people in care for a diagnosis they report, 24% with ME/CFS and 41% with endometriosis have its code, against 85% for hypertension.

UNOM turns data the government already collects into public tools for nine invisible illnesses in five families (infection-associated chronic illness, autoimmune, chronic pain, women's health, mental health). From All of Us and public CDC, Census, and CMS data, we built nine machine learning models and made them free to download. The same models serve everyone: clinicians see flagged illnesses with their evidence inside the health record, patients use them in a free app, policy makers see who goes uncoded in the Atlas, and developers can build them into their own software.

Patients answer validated questions, in English or Spanish, on their own device, then print or download a summary that brings their clinician up to speed. Their answers can change the estimate substantially: in our Long COVID example, from 6.1 to 15.4 times the average. Patient advocates told the sprint that patients are disbelieved and too sick to argue, so the evidence goes to the clinician, and the patient approves the summary before sharing it.

Data, Tech & Evidence Driven Approach

UNOM connects U.S. government data no single source links. The NIH All of Us Research Program links survey answers to health records for 161,480 adults, showing who reports a diagnosis the record never coded, and follows 23,790 adults with COVID-19 across 54 record sites. Fitbit data add activity and sleep. CDC surveys and Lyme surveillance give prevalence and cases, Census gives county population, and CMS Medicare payments price the diagnostic odyssey.

The AI is nine open models sharing one scorer. A model trained on codes learns who gets coded, so eight of ours learn from patients' own reports of a clinician's diagnosis. On all eight they rank uncoded patients higher than code-trained models (AUROC 0.74 vs 0.67 for ME/CFS, 0.73 vs 0.64 fibromyalgia, 0.61 vs 0.53 endometriosis, 0.82 vs 0.65 lupus, 0.79 vs 0.54 anxiety). The Long COVID model, trained on later codes, reached AUROC 0.67, and 0.66 at held-out sites.

Four papers, submitted to JAMIA, JAMA Network Open, the Journal of Biomedical Informatics, and Applied Clinical Informatics, carry the evidence: (1) how much Long COVID goes uncoded and how early records predict its code, (2) how often ten self-reported diagnoses are coded by sex, age, race and ethnicity, and health system, (3) why patients' reports find uncoded patients better than codes do, and (4) how the system was built and tested.

The technology is built on health IT standards: HL7 FHIR R4 records coded in ICD-10-CM, LOINC, RxNorm, SNOMED CT, and CPT, SMART on FHIR to open inside an EHR, CDS Hooks to send cards to an EHR, and FHIR questionnaires from the patient app. One JavaScript scoring engine runs in the apps and the CDS Hooks service, and matches a Python reference on all 180 expected outputs.

Execution, Efficiency & Delivery:

UNOM can be deployed today without new research or new infrastructure. The clinician app installs in an EHR as a standard SMART on FHIR app and already runs in the public SMART sandbox (registered with Epic and athenaOne, sandbox access pending). A CDS Hooks service sends the same cards to an EHR. Charts are scored in the browser and patient answers stay on the patient's device, so no health data is stored on our servers, which shortens a hospital's security review.

It delivers value at the point of care. Each card turns a chart into a decision: the evidence, the criteria to check, and a next step, such as a post-COVID clinic referral, CDC's two-step Lyme test, or an ANA test for lupus. Earlier recognition can shorten a costly odyssey: before their Long COVID was coded, All of Us patients had 6.6 more outpatient and 0.4 more emergency visits than matched post-COVID patients, about $900 each, and early chart data flagged them a median of 55 days before the code. A two-step screen costs about $116 of clinician time per patient found.

It reaches every user, even before a hospital adopts it. Patients use the patient app in any browser, with no account or install, and take its summary to their visit. Anyone can look up an illness in the public Atlas. Developers can download the free model files and the scorer and build them into their own tools. Automated tests check every result.

Scalability, Reuse & Societal Impact:

Adding a new illness to UNOM is easy. It takes one new model file, built with the same pipeline from the same list of standard codes, and a clinician card for it. The same scorer then runs it in every app. POTS or post-treatment Lyme disease could be next.

UNOM runs on the standards that certified EHRs must support under the 21st Century Cures Act: SMART on FHIR, USCDI data, and CDS Hooks. Because the models are fit from aggregate counts, the VA, IHS, CMS, or any research network could build and share models the same way without moving patient data.

Everything is free to reuse. Developers can download the model files with their model cards, the scoring library, the CDS Hooks service, and test patients, and build UNOM into their own tools. Policy makers and researchers can use the Atlas to see who goes uncoded for each illness, by state, sex, age, and race and ethnicity.

Its main societal impact is giving patients a voice. In the patient app, people check their symptoms against published criteria and see which health record patterns our models look for. Their answers to validated questionnaires update the model's estimate, so their own account counts next to the chart. They print or download a summary in a standard format clinic software can read and share it with their clinician. They come to a hard visit informed and prepared, without retelling their history.

Public (9)
You will need to login to post a comment
Uri Kartoun 2 weeks ago

The line that stood out to me is that a model trained on codes learns who gets coded, so your eight models learn from patients’ own reports instead. Two papers in my library support that from the other direction.

I work on turning published Lyme and Long COVID papers into runnable simulators. Simulator 24 covers a national health-records study (N3C, Pfaff et al., BMC Med 2023) of 33,782 patients given the Long COVID code U09.9. Compared with all COVID-19-positive patients at the same sites, the coded group was 66.9% female (vs 55.6%) and 71.8% White (vs 61.1%), and the authors suggest the skew reflects who gets diagnosed and coded, not the true distribution of Long COVID (chart attached). Simulator 13, a 58-hospital study, found Long COVID in 16.3% of COVID-19 patients with an AI method against under 7% by diagnosis codes. These measure different things from your coded-share numbers, so I’d read them as complementary, not contradictory.

My entry is here if you’re curious: https://invisibleillness.crowdicity.com/post/896844

Thanks for making the models free to download. It was a pleasure reading your work.

Reply 1

Amir Asiaee 1 week ago

Thanks, Uri.

Pfaff et al. is a good companion to our numbers. They compare people coded U09.9 with everyone who tested positive at the same sites, and we compare coded people with those who report the diagnosis, or for Long COVID with Household Pulse prevalence. The denominators differ, but the lesson is the same. The code follows access to diagnosis as much as the illness itself. The 16.3% against under 7% result fits what we see in All of Us too, where about 1 in 9 adults who ever had Long COVID carry the code.

One possible link with your work. Your simulators flag Long COVID figures that come from codes or claims as likely undercounts. Our Atlas has the coded share for each condition by state, sex, age, and race and ethnicity, so it could tell a simulator roughly how much a code-based figure misses. It's public at undiagnosednomore.org/atlas.

Reply 0

Uri Kartoun 1 week ago

Thank you, Amir. The point about denominators is helpful, and I agree that the code follows access to diagnosis as much as the illness. You've described my side correctly: the simulators flag a code-based figure as a likely undercount but don't say by how much. A coded share by state, age and group is the kind of input that could turn that flag into a stated range, so the Atlas is worth a careful look. I'll read how it's built and what it covers before saying more. Thanks for pointing to it.

Reply 1

Jisung Chang 2 weeks ago (edited)

Amir, impressive work — I especially like how UNOM addresses the gap between what patients experience and what actually becomes visible in the health record, while connecting the same models across patients, clinicians, policy makers, and developers.

I also see an interesting potential connection with Lyme AI-ert. Lyme AI-ert focuses upstream on geographic and temporal Lyme signals and early community awareness, while UNOM works further downstream to identify patients whose patterns may be missed in the clinical record.

It would be interesting to explore whether community-level risk and temporal signals could provide additional context alongside UNOM’s patient- and EHR-level signals.

If useful, you can explore Lyme AI-ert here, including interactive maps of weekly alerts and predicted annual Lyme cases:
https://invisibleillness.crowdicity.com/post/896784

Great work, and I’d be interested in exploring potential connections between our approaches in the future.

Reply 1

Sulamit Berkovits 1 week ago

Thank you for raising this connection, Jisung. I really like the idea of community level risk and temporal signals as context, not as a replacement for what a patient reports. A clinician card could read very differently if it knew, for example, that reports in a certain area or season were rising, while still keeping the individual chart entries and patient reported experience separate and visible. I am curious how you would keep that context from tipping the card toward a geographic pattern when the individual evidence points elsewhere. Thanks again for linking the two approaches so clearly.

Reply 0

Amir Asiaee 1 week ago

To clarify: In our Lyme card, county incidence never flags a chart by itself. A chart goes to Needs review if it has two kinds of Lyme findings, or one finding in a moderate- or high-incidence county. So a chart with no findings stays unflagged even in a high-incidence county, and a chart with two findings is flagged even in a low-incidence one. The county sentence is shown apart from the chart findings, so the clinician can tell which is which.

Reply 0

Amir Asiaee 1 week ago

Thanks, Jisung. Our Lyme card already does a small version of this. When a chart has a Lyme finding, such as a tick bite visit, a rash, doxycycline, or facial palsy, the card adds the county's reported incidence from CDC surveillance as exposure context. It uses 2015 to 2019 averages, so a weekly county alert like yours would be more current. Your alerts describe community priority rather than individual risk, and that's how we'd use them too, as context next to the chart and never as a reason to flag on their own. Glad to talk after the sprint.

Reply 0

Sulamit Berkovits 1 week ago

Thank you for the clarity of this work, especially the distinction you draw between who gets coded and who actually has the condition. The contrast you report — only 24% with ME/CFS, 39% with Lyme disease, and 41% with endometriosis carrying their code, versus 85% for hypertension — makes the case for a flagging system much more concretely than a general claim about underdiagnosis.

I have two questions about how the clinician card handles evidence in practice. First, when dated chart entries point in opposite directions, does the card surface that conflict, or does it net those entries into a single band? And can the clinician still see the chart-only estimate separately from the patient-updated estimate — for example, in your Maria example, the move from 6.1 times the average on chart data alone to 15.4 times with patient answers — rather than only the final combined number?

Second, with Long COVID at AUROC 0.67 and endometriosis at 0.61 in your reported results, and the current testing still synthetic with SMART sandbox access pending, what “needs review” threshold are you envisioning for a real primary-care panel, and what flag rate and false-flag workup would that imply? I am also curious how a flag would be audited months later, given your versioned model files with model cards and hashes — would the audit trail preserve which model version, which entries, and which likelihood ratios produced that specific card?

Reply 1

Amir Asiaee 1 week ago

Thank you, Sulamit.

The card lists every dated entry with its own arrow, up or down, and a bar for how much it moves the odds, so entries that pull in opposite directions both show. The band is the net of all of them, and the clinician can see what went into it.

The chart-only and patient-updated estimates are shown separately. Maria's card reads 15.4 times the average with her answers and 6.1 times from her chart alone, and her answers get their own row with the likelihood ratio they apply (×2.83 for her RECOVER score).

Needs review currently lists conditions in the Elevated band (1.5 to 3 times the average) or the High band (above 3 times). Those are display bands, not a tuned operating point. For a real panel I'd set the cutoff by how many charts the clinic can review. For Long COVID, flagging the top 5% of scores at day 90 catches 19% of the patients who later get the code, and 4.3% of flags get it. That's measured against a code that misses most cases, so it understates true positives, but it's still a lot of review. That's why the Atlas cost calculator models a two-step screen, where flagged patients answer the questionnaire first and a clinician reviews only those who pass. With the default assumptions that's 200 chart reviews per 10,000 patients, about 17 later-coded patients found, and $116 of clinician time per patient found. At a new site I'd run the models silently first to measure the local flag rate.

For auditing, every card names its model and version, for example unm-longcovid 1.2.0. Each version is a fixed file with a SHA-256 hash in our public registry, and the scorer is deterministic and matches a Python reference on all 180 expected outputs. So the chart as of that date plus the model version reproduces the card exactly, including the entries and likelihood ratios. We don't store anything on our servers, though, so the log itself would have to live on the EHR side.

Reply 0

Sulamit Berkovits 1 week ago

Thank you, Amir, this is the kind of detail I really appreciate. Dated entries on a clinician card, and being clear about bands and about which model or version produced a result, that feels like the difference between something you can actually audit and something you just have to trust. And your point that county incidence alone never flags a chart by itself, that is an important guardrail, because a population signal is not a personal finding. I guess what I am curious about is how that auditing shows up to the clinician in the room, like whether they can see at a glance what changed between versions, or whether they would have to dig for it. Thanks for going that deep on it.

Reply 0

Amir Asiaee 1 week ago

Not at a glance today. The card names the model and version that produced it, but it doesn't show what changed from the previous version. The model files would/could allow a side-by-side view, scoring the same chart under both versions and showing which entries moved and by how much. We haven't built it yet.

Reply 0

Sulamit Berkovits 1 week ago

That honesty is actually reassuring, that the card says what version produced it even before the comparison view exists. What sticks with me is that the version number alone can’t tell a clinician whether today’s flag means something different from last week’s. The interesting unit isn’t really the version, it’s which entries moved and by how much. A flag that crossed a band between versions feels like the one that most needs its explanation attached, because that’s where a clinician might act differently without realizing the ground shifted. When the side by side does get built, would it live right on the card where they already are, or somewhere they’d have to go looking for it?

Reply 0

Sulamit Berkovits 1 week ago

The flag is not the hard part here. The hard part is what happens to it after the visit ends.

I would test UNOM in one primary care clinic, for one condition to start, with a named owner, a nurse navigator, not the flag itself, responsible for every flagged patient already booked for a visit. The 90 days would test workflow and plumbing, not model accuracy: does the SMART launch actually open in the production EHR, rather than the sandbox? Do patient summaries filed as FHIR files actually land in the chart, or does printed paper get handed over and lost? Are referrals and tests ordered from a card completed, or orphaned? Do patients already worked up elsewhere get duplicate testing? I would measure weekly: launch and open failures, summaries filed versus handed over but never filed, flags with a documented next step within 14 days, referral and test completion and time to completion, and dismissals by reason. And I would set a pause rule in advance: if summaries are routinely not filed, or flags routinely have no owner, stop. In real deployment the likeliest break is not the score, but a flag with no accountable next step and no closed loop back to coding correction or rule out.

Reply 1

Amir Asiaee 1 week ago

Agreed. The score is the easy part to test. Your weekly counts are close to what we'd want from a first clinic, especially launch failures in the production EHR and summaries that actually get filed. I'd add one more count, flags that end in a new code versus flags that are ruled out, because that tells you whether the record actually gets corrected.

Reply 0

Sulamit Berkovits 1 week ago

That new code versus ruled out count is the one I'd been circling without naming it. The closed loop back to the record was the whole worry in my original comment, and this is what it looks like measured. The ruled out side interests me just as much, because a rule out is also a finding, and if it only lives in a dismissal note instead of a code, the next version trains on the same gap. Do you picture the navigator writing the new code, or does that stay with the clinician? And I'd love to know whether a ruled out flag earns anything in the record at all, or whether it just quietly disappears.

Reply 0

Hi Undiagnosed, How does your team approach cases where a patient reports a diagnosis or persistent symptoms but the clinical record doesn't contain the corresponding code, without creating additional work for the patient?

Reply 0

Pranisha Karki 1 week ago

Hi Amir, it's great that the clinician app, patient app and atlas are all public, with every patient on the site synthetic. For the nine conditions, how do the All of Us models perform for groups that are less represented in the data, and do you report accuracy separately for those groups?

Reply 0

Scott Savitt 6 days ago

Hi Amir, thanks for sharing. The gap between how often people report these conditions and how often they're actually coded is striking. On the patient side, how do you explain a flag to someone so it doesn't read like a diagnosis, or like they're being told something is wrong with them?

Reply 0

Pranisha Karki 5 days ago

Who has found the Atlas most useful so far: clinicians, researchers or policy staff?

Reply 0

Pranisha Karki 1 day ago

Thank you for sharing these coding figures from All of Us. For your reported 0.79 ratio for women versus men, could you share the denominator (how many people reporting Long COVID in each group), the cohort dates, and the uncertainty range around it? That would help readers understand how to interpret your reported figure.

Reply 0