Interpretable ML in Clinical Decision-Making
Keywords:
Artificial Intelligence in Healthcare, Clinical Decision Support Systems, Explainable Artificial Intelligence (XAI), Model Transparency in Medicine, Interpretability of Machine Learning Models, Accountability in Clinical AI, Risk Prediction and Prognostic Modeling, Black-Box Model Limitations, Human-Centered AI Design, Trustworthy Medical AI, Ethical AI in Healthcare, Model Explainability Frameworks, Data-Driven Clinical Risk Assessment, AI Governance in Health Systems.Abstract
Artificial intelligence has the potential to augment clinical decision making. By learning patterns of risk and disease directly from empirical data, AI methods offer one solution to the difficulty health care professionals face in considering ever-increasing amounts of information. Clinicians making a medical decision for a patient want not only an accurate estimate of the risks associated with their patient's disease or treatment options but also an understanding of the reasoning behind these risks. This desire for explanation drives the growing interest in explainability in AI, particularly in AI for health care.
Explainable Artificial Intelligence (XAI) is defined as methods that generate new AI models for which the behaviour can be understood, directly or indirectly, by humans. The concept of human understanding encompasses three different levels – transparency, interpretability and accountability. The heart of the concern for transparency in AI is the incomprehensibility of the learned representations, the “black box” nature of the complex function learned from the training data.
References
1. Arjunan, G. (2021). Implementing explainable AI in healthcare: Techniques for interpretable machine learning models in clinical decision-making. International Journal of Scientific Research and Management, 9(5), 597–603.
2. Chakrobartty, S., & El-Gayar, O. (2021). Explainable artificial intelligence in the medical domain: A systematic review. Proceedings of the Americas Conference on Information Systems.
3. Inala, R. (2023). AI-powered investment decision support systems: Building smart data products with embedded governance controls. Journal for ReAttach Therapy and Developmental Diversities, 6(10), 2251-2266.
4. Fang, H. S. A., Tan, N. C., Tan, W. Y., Oei, R. W., Lee, M. L., & Hsu, W. (2021). Patient similarity analytics for explainable clinical risk prediction. BMC Medical Informatics and Decision Making, 21, 207.
5. Loh, H. W., Ooi, C. P., Seoni, S., Barua, P. D., Molinari, F., & Acharya, U. R. (2022). Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011–2022). Computer Methods and Programs in Biomedicine, 226, 107161.
6. Yang, C. C. (2022). Explainable artificial intelligence for predictive modeling in healthcare. Journal of Healthcare Informatics Research, 6(2), 228–239.
7. Kolla, T. (2024). Graph Neural Networks for HCC Risk Adjustment and Interoperability. International Journal of Science, Research and Technology, 7(6), 13244-13255.
8. Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., Madai, V. I., Mühleisen, L., Shavlokhova, V., Zicari, R. V., & others. (2022). To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems. PLOS Digital Health, 1(2), e0000016.
9. Combi, C., Amico, B., Bellazzi, R., Holzinger, A., Moore, J. H., Zitnik, M., & Holmes, J. H. (2022). A manifesto on explainability for artificial intelligence in medicine. Artificial Intelligence in Medicine, 133, 102423.
10. Inala, R. Designing Scalable Technology Architectures for Customer Data in Group Insurance and Investment Platforms.
11. Pierce, R. L., Van Biesen, W., Van Cauwenberge, D., Decruyenaere, J., & Sterckx, S. (2022). Explainability in medicine in an era of AI-based clinical decision support systems. Frontiers in Genetics, 13, 903600.
12. Kempt, H., Heilinger, J.-C., & Nagel, S. K. (2022). Relative explainability and double standards in medical decision-making. Ethics and Information Technology, 24, 20.
13. Di Martino, F., & Delmastro, F. (2023). Explainable AI for clinical and remote health applications: A survey on tabular and time series data. Artificial Intelligence Review, 56, 5261–5315.
14. Kolla, S. K., & Reddy, V. A. R. (2024). Evaluating Cloud-Native vs. Hybrid Architectures for Health Benefit Administration Systems. International Journal of Medical Toxicology and Legal Medicine, 27(5), 1042-1053.
15. Xu, Q., Xie, W., Liao, B., Hu, C., Qin, L., Yang, Z., Xiong, H., Lyu, Y., Zhou, Y., & Luo, A. (2023). Interpretability of clinical decision support systems based on artificial intelligence from technological and medical perspective: A systematic review. Journal of Healthcare Engineering, 2023, 9919269.
16. Wysocki, O., Davies, J. K., Vigo, M., Armstrong, A. C., Landers, D., Lee, R., & Freitas, A. (2023). Assessing the communication gap between AI models and healthcare professionals: Explainability, utility and trust in AI-driven clinical decision-making. Artificial Intelligence, 316, 103839.
17. Anjara, S. G., Janik, A., Dunford-Stenger, A., Mc Kenzie, K., Collazo-Lorduy, A., Torrente, M., & Provencio, M. (2023). Examining explainable clinical decision support systems with think aloud protocols. PLOS ONE, 18(9), e0291443.
18. Davuluri, P. S. L. (2023). AI-Augmented Sanctions Screening: Enhancing Accuracy and Latency in Real Time Compliance Systems. AI-Augmented Sanctions Screening: Enhancing Accuracy and Latency in Real Time Compliance Systems (December 15, 2023).
19. Jin, W., Li, X., Fatehi, M., & Hamarneh, G. (2023). Guidelines and evaluation of clinical explainable AI in medical image analysis. Medical Image Analysis, 84, 102684.
20. Islam, M. R., Ali, M. S., & others. (2023). Application of explainable artificial intelligence in medical health: A systematic review of interpretability methods. Informatics in Medicine Unlocked, 40, 101286.
21. Sarker, I. H. (2023). Machine learning: Algorithms, real-world applications and research directions. SN Computer Science, 2, 160.
22. Zhang, J., & Zhang, Z.-m. (2023). Ethics and governance of trustworthy medical artificial intelligence. BMC Medical Informatics and Decision Making, 23, 7.
23. Kolla, S. K., & Mangalampalli, B. M. (2024). Edge-Based Deep Learning Systems for Point-of-Care Diagnostic Intelligence. Journal of Neonatal Surgery, 13(1), 2387-2399.
24. Feng, T., Noren, D. P., Kulkarni, C., Mariani, S., Zhao, C., Ghosh, E., Swearingen, D., Frassica, J., McFarlane, D., & Conroy, B. (2023). Machine learning-based clinical decision support for infection risk prediction. Frontiers in Medicine, 10, 1213411.
25. Trottet, C., Vogels, T., Keitel, K., Kulinkina, A. V., Tan, R., Cobuccio, L., & others. (2023). Modular clinical decision support networks (MoDN)—Updatable, interpretable, and portable predictions for evolving clinical environments. PLOS Digital Health, 2(6), e0000108.
26. Del Gaizo, J., & others. (2023). Red Flag/Blue Flag visualization of a common CNN for text classification. JAMIA Open, 6(1), ooac112.
27. Baniecki, H., Sobieski, B., Szatkowski, P., Bombinski, P., & Biecek, P. (2024). Interpretable machine learning for time-to-event prediction in medicine and healthcare. Artificial Intelligence in Medicine, 154, 103026.
28. Gottimukkala, V. R. R. (2024). Federated Learning Approaches for Fraud Detection in International Payment Systems. https://www. jisem-journal. com/download/118_JISEM. pdf.
29. Liu, S., McCoy, A. B., Peterson, J. F., Lasko, T. A., Sittig, D. F., Nelson, S. D., Andrews, J., Patterson, L., Cobb, C. M., Mulherin, D., Morton, C. T., & Wright, A. (2024). Leveraging explainable artificial intelligence to optimize clinical decision support. Journal of the American Medical Informatics Association, 31(4), 968–974.
30. Nasarian, E., Alizadehsani, R., Acharya, U. R., & Tsui, K.-L. (2024). Designing interpretable ML system to enhance trust in healthcare: A systematic review to proposed responsible clinician-AI-collaboration framework. Information Fusion, 108, 102412.
31. Gerdes, A. (2024). The role of explainability in AI-supported medical decision-making. Discover Artificial Intelligence, 4, Article 20.
32. Rosenbacke, R., Melhus, Å., McKee, M., & Stuckler, D. (2024). How explainable artificial intelligence can increase or decrease clinicians’ trust in AI applications in health care: Systematic review. JMIR AI, 3, e53207.
33. Inala, R., & Somu, B. (2024). Agentic ai in retail banking: Redefining customer service and financial decision-making. Journal of Artificial Intelligence and Big Data Disciplines, 1(1), 1-19.
34. Freyer, N., Groß, D., & Lipprandt, M. (2024). The ethical requirement of explainability for AI-DSS in healthcare: A systematic review of reasons. BMC Medical Ethics, 25, 104.
35. Sirocchi, C., Bogliolo, A., & Montagna, S. (2024). Medical-informed machine learning: Integrating prior knowledge into medical decision systems. BMC Medical Informatics and Decision Making, 24, 186.
36. Aziz, N. A., Manzoor, A., Qureshi, M. D. M., Qureshi, M. A., & Rashwan, W. (2024). Explainable AI in healthcare: Systematic review of clinical decision support systems. medRxiv.
37. Nasarian, E., Alizadehsani, R., Acharya, U. R., & Tsui, K.-L. (2024). Designing interpretable ML system to enhance trust in healthcare: A systematic review to proposed responsible clinician-AI-collaboration framework. Information Fusion, 108, 102412.
38. Aravindkumar, A., Ramadoss, M., Fakhruddin Ahmed, S. A., Sampath, V., & Lakshminarayanan, K. (2024). Explainable AI in healthcare: A systematic review of XAI use cases in imaging, diagnostics, and rehabilitation. Journal of Medical Systems.
