Clinical AI systems are only as good as the data they are trained on, and according to experts, that data is fundamentally flawed. In a recent discussion, researchers and healthcare professionals highlighted a critical issue: electronic medical records are not designed to capture accurate biological data about patients.
"We do a terrible job of actually measuring human biology, and that data just does not exist really anywhere," one expert noted.
Instead, medical records serve primarily as billing and administrative tools. As another source explained, "What people don't know is medical records are not actually designed to be data capture systems that reflect your clinical reality and your health."
This misalignment leads to systematic bias in the data used to train AI. Records often exaggerate a patient's condition to maximize reimbursement. "There is a systematic bias in all of the data to make you look a little bit more sick than you do in order to increase revenue," the expert added.
Consequently, clinical AI models trained on such data may learn to over-diagnose or recommend unnecessary treatments, potentially harming patients. The challenge underscores the need for better data collection methods that truly measure patient biology rather than billing codes.