Closed large language models (LLMs), developed by commercial or institutional entities, rarely disclose their training datasets. This lack of transparency conceals the social and clinical biases embedded within them.<\/p>\n\n\n\n
According to The Lancet Digital Health<\/em>, every health dataset carries limitations that can encode inequities into AI systems.<\/p>\n\n\n\n Underrepresentation of women, ethnic minorities, or low-income groups reduces algorithmic accuracy and perpetuates inequality in care.<\/p>\n\n\n\n Bias can also arise from missing or incomplete data. For instance, patients from underserved regions may undergo fewer diagnostic tests, leading algorithms to mistakenly label them \u201chealthy\u201d when they are not.<\/p>\n\n\n\n A transparent model, whether open-source or simply well-documented, allows clinicians, regulators, and patients to understand how it works.<\/p>\n\n\n\n <\/p>\n\n\n\n This framework compares closed LLMs, which rely on external commercial servers, with open LLMs that run within local healthcare environments. While open models require greater technical expertise, they offer more control and keep sensitive patient data local.<\/p>\n\n\n\n The Frontiers in Digital Health<\/em> \u201ctrustworthy AI reality check\u201d found that fewer than one-third of CE-certified radiology AIs in Europe provided basic documentation on training data, consent procedures, or limitations. Without such information, healthcare organisations cannot accurately assess the risk of bias.<\/p>\n\n\n\n Transparency, as emphasised by the Institute for Healthcare Improvement<\/em>, builds trust. Patients should be informed when AI is used in their care, and clinicians must receive practical training to use these tools safely and ethically.<\/p>\n\n\n\n Even the most sophisticated AI cannot transcend the biases inherent in the system that generated its data. Researchers at the Mayo Clinic describe two intertwined forms of bias:<\/p>\n\n\n\n Bias can emerge at any of the six stages of AI development, from conception and data collection to clinical deployment and post-deployment surveillance. Each stage introduces distinct risks, including representation bias, algorithmic bias, and feedback loop bias, all of which can undermine fairness and equity in healthcare.<\/p>\n\n\n\n Recent meta-analyses have revealed that generative AI models can match the diagnostic accuracy of non-expert physicians but still lag behind specialists, achieving an accuracy rate of around 52%. This positions AI as a clinical aid, not a replacement. However, many studies fail to disclose their training data provenance, raising ongoing concerns about reproducibility and fairness.<\/p>\n\n\n\n The five-step evaluation process outlines a practical path to building trustworthy AI:<\/p>\n\n\n\nWhy Transparency Matters in AI Development<\/strong><\/h2>\n\n\n\n
![\"\"](\"https:\/\/orionhealth.com\/wp-content\/uploads\/Vital-Signs-with-Tom-Varghese-Closed-vs.-open-approach-of-deploying-LLMs-for-clinical-applications.svg\")
Source: Dennst\u00e4dt et al., NPJ Digital Medicine (2025)<\/em><\/figcaption><\/figure>\n\n\n\nRecognising bias across the AI model life cycle<\/strong><\/h2>\n\n\n\n
\n
![\"\"](\"https:\/\/orionhealth.com\/wp-content\/uploads\/Vital-Signs-with-Tom-Varghese-The-AI-model-life-cycle-and-common-biases-across-each-phase.svg\")
Source: Hasanzadeh et al., NPJ Digital Medicine (2025)<\/em><\/figcaption><\/figure>\n\n\n\nEvaluating and mitigating bias in large language models<\/strong><\/h2>\n\n\n\n
![\"\"](\"https:\/\/orionhealth.com\/wp-content\/uploads\/Vital-Signs-with-Tom-Varghese-Five-step-process-for-large-language-model-evaluation.svg\")
Source: Templin et al., NPJ Digital Medicine (2025)<\/em><\/figcaption><\/figure>\n\n\n\n