Data Engineering for Longitudinal EHR Analytics
Keywords:
AI, healthcare, clinical data engineering, EHR analytics, feature engineering, health decision systems, risk prediction, data interoperability .Abstract
Healthcare systems generate large volumes of Electronic Health Record (EHR) data. Translating this information into clinical intelligence that supports routine decision-making requires AI-driven systems grounded in sound Clinical Data Engineering principles. This work presents three such systems. First, a Clinical Data Engineering framework built on Application Programming Interfaces for hospitals, structured around the Seven Interoperable Levels of Data Standardization; API-based data-sharing services normalize clinical data for secondary use and improve its suitability for clinical analytics. Second, a high-quality risk prediction and patient stratification approach for patients undergoing elective cardiac surgery, achieved by integrating log data from an EHR-based Clinical Decision Support System with information drawn directly from the EHR; a gradient-boosted tree algorithm achieved benchmark performance for this task. Third, an AI-driven feature engineering system that standardizes clinical information for EHR analytics and identifies attributes associated with patient outcomes, using automated data preparation and feature extraction pipelines that assemble the complete clinical dataset directly from the database while supporting unsupervised feature selection.
Effective EHR analytics is a prerequisite for successful precision healthcare decision systems, and validated stratification and risk-prediction methods are fundamental building blocks of that concept. The strategies proposed here for stratifying patients undergoing elective cardiac surgery and forecasting their outcomes point toward further development in this direction — particularly the integration of analytic log data from EHR-based Clinical Decision Support Systems with the broader information available in the EHR.
References
1. Arbet, J., Brokamp, C., Meinzen-Derr, J., Trinkley, K. E., & Spratt, H. M. (2020). Lessons and tips for designing a machine learning study using EHR data. Journal of Clinical and Translational Science, 5(1), e21.
2. Ghassemi, M., Naumann, T., Schulam, P., Beam, A. L., Chen, I. Y., & Ranganath, R. (2020). A review of challenges and opportunities in machine learning for health. AMIA Joint Summits on Translational Science Proceedings, 2020, 191–200.
3. Lee, D., Jiang, X., & Yu, H. (2020). Harmonized representation learning on dynamic EHR graphs. Journal of Biomedical Informatics, 106, 103426.
4. A, R., Maheswari, N. U., Umamaheswari, P., Josephson, P. J., Mangalampalli, B. M., & Gudada, P. V. (2026). Tree-Structured Regularization Driven CNN Model for Liver Disease Diagnosis and Classification Using Ultrasound Imaging. In 2026 International Conference on Intelligent and Sustainable Electronics & Computing Technologies (INSECT) (pp. 1–6). IEEE. 2026 International Conference on Intelligent and Sustainable Electronics & Computing Technologies (INSECT). https://doi.org/10.1109/insect68872.2026.11663742
5. Thapa, C., & Camtepe, S. (2020). Precision health data: Requirements, challenges and existing techniques for data security and privacy. Computers in Biology and Medicine, 129, 104130.
6. Wang, Y., Zhao, Y., Therneau, T. M., Atkinson, E. J., Tafti, A. P., Zhang, N., Amin, S., Limper, A. H., Khosla, S., & Liu, H. (2020). Unsupervised machine learning for the discovery of latent disease clusters and patient subgroups using electronic health records. Journal of Biomedical Informatics, 102, 103364.
7. Steinberg, E., Jung, K., Fries, J. A., Corbin, C. K., Pfohl, S. R., & Shah, N. H. (2021). Language models are an effective representation learning technique for electronic health record data. Journal of Biomedical Informatics, 113, 103637.
8. Xie, F., Yuan, H., Ning, Y., Ong, M. E. H., Feng, M., Hsu, W., Chakraborty, B., & Liu, N. (2022). Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies. Journal of Biomedical Informatics, 126, 103980.
9. Xiao, X., Wei, G., Zhou, L., Pan, Y., Jing, H., Zhao, E., & Yuan, Y. (2021). Treatment initiation prediction by EHR mapped PPD tensor based convolutional neural networks boosting algorithm. Journal of Biomedical Informatics, 120, 103840.
10. Bhandari, P., Prithviraj, J., Saxena, R., Kumar, D., Mangala, N., & Chourasiya, P. (2026). Harris Hawks Optimization–Based Deep Learning for Resource-Aware Cloud–Fog Computing. In 2026 5th OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 5.0 (pp. 1–6). IEEE. 2026 5th OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 5.0. https://doi.org/10.1109/otcon69332.2026.11629982
11. Mullin, S., Zola, J., Lee, R., Hu, J., MacKenzie, B., Brickman, A., Anaya, G., Sinha, S., Li, A., & Elkin, P. L. (2021). Longitudinal K-means approaches to clustering and analyzing EHR opioid use trajectories for clinical subtypes. Journal of Biomedical Informatics, 122, 103889.
12. Rogers, J. R., Hripcsak, G., Cheung, Y. K., & Weng, C. (2021). Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method. Journal of Biomedical Informatics, 119, 103822.
13. Tang, Z., Yu, Y., Ng, K., Sow, D., Hu, J., & Mei, J. (2021). Disease network delineates the disease progression profile of cardiovascular diseases. Journal of Biomedical Informatics, 115, 103686.
14. Ma, X., Imai, T., Shinohara, E., Kasai, S., Kato, K., Kagawa, R., & Ohe, K. (2021). EHR2CCAS: A framework for mapping EHR to disease knowledge presenting causal chain of disorders—Chronic kidney disease example. Journal of Biomedical Informatics, 115, 103692.
15. Ulrich, H., Behrend, P., Wiedekopf, J., Drenkhahn, C., Kock-Schoppenhauer, A.-K., & Ingenerf, J. (2021). Hands on the Medical Informatics Initiative Core Data Set—Lessons learned from converting the MIMIC-IV. Studies in Health Technology and Informatics, 283, 119–126.
16. Giesa, N., Heeren, P., Klopfenstein, S., Flint, A., Agha-Mir-Salim, L., Poncette, A., Balzer, F., & Boie, S. (2022). MIMIC-IV as a clinical data schema. Studies in Health Technology and Informatics, 294, 559–560.
17. Hobensack, M., Song, J., Scharp, D., Bowles, K. H., & Topaz, M. (2023). Machine learning applied to electronic health record data in home healthcare: A scoping review. International Journal of Medical Informatics, 170, 104978.
18. Amirahmadi, A., Ohlsson, M., & Etminani, K. (2023). Deep learning prediction models based on EHR trajectories: A systematic review. Journal of Biomedical Informatics, 144, 104430.
19. Malhotra, A., Niharika, D. S., Mangalampalli, B. M., Nema, S., Gupta, Z., & Kriplani, K. (2026, June). Distributed and Edge AI for Continuous and Context-Aware Healthcare Intelligence. In 2026 5th OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 5.0 (pp. 1-6). IEEE.
20. Carrasco-Ribelles, L. A., Llanes-Jurado, J., Gallego-Moll, C., Cabrera-Bean, M., Monteagudo-Zaragoza, M., Violán, C., & Zabaleta-Del-Olmo, E. (2023). Prediction models using artificial intelligence and longitudinal data from electronic health records: A systematic methodological review. Journal of the American Medical Informatics Association, 30(12), 2072–2082.
21. Caruana, A., Bandara, M., Musial, K., Catchpoole, D., & Kennedy, P. J. (2023). Machine learning for administrative health records: A systematic review of techniques and applications. Artificial Intelligence in Medicine, 144, 102642.
22. Noaeen, M., Amini, S., Bhasker, S., Ghezelsefli, Z., Ahmed, A., Jafarinezhad, O., & Shakeri Hossein Abad, Z. (2023). Unlocking the power of EHRs: Harnessing unstructured data for machine learning-based outcome predictions. 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, 1–4.
23. Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., Lehman, L.-W. H., Celi, L. A., & Mark, R. G. (2023). MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 10(1), 1.
24. Bennett, A. M., Ulrich, H., van Damme, P., Wiedekopf, J., & Johnson, A. E. W. (2023). MIMIC-IV on FHIR: Converting a decade of in-patient data into an exchangeable, interoperable format. Journal of the American Medical Informatics Association, 30(4), 718–725.
25. Ryu, A. J., Ayanian, S., Qian, R., Core, M. A., Heaton, H. A., Lamb, M. W., Parikh, R. S., Boyum, J. P., Garza, E. L., Condon, J. L., & Peters, S. G. (2023). A clinician's guide to running custom machine-learning models in an electronic health record environment. Mayo Clinic Proceedings, 98(3), 445–450.
26. Deimazar, G., & Sheikhtaheri, A. (2023). Machine learning models to detect and predict patient safety events using electronic health records: A systematic review. International Journal of Medical Informatics, 180, 105246.
27. Swinckels, L., Bennis, F. C., Ziesemer, K. A., Scheerman, J. F. M., Bijwaard, H., de Keijzer, A., & Bruers, J. J. (2024). The use of deep learning and machine learning on longitudinal electronic health records for the early detection and prevention of diseases: Scoping review. Journal of Medical Internet Research, 26, e48320.
28. Nasarudin, N. A., Al Jasmi, F., Sinnott, R. O., Zaki, N., Al Ashwal, H., Mohamed, E. A., & Mohamad, M. S. (2024). A review of deep learning models and online healthcare databases for electronic health records and their use for health prediction. Artificial Intelligence Review, 57, 249.
29. Lu, Y., Tong, J., Chubak, J., Lumley, T., Hubbard, R. A., Xu, H., & Chen, Y. (2024). Leveraging error-prone algorithm-derived phenotypes: Enhancing association studies for risk factors in EHR data. Journal of Biomedical Informatics, 157, 104690.
30. Hao, B., Hu, Y., Adams, W. G., Assoumou, S. A., Hsu, H. E., Bhadelia, N., & Paschalidis, I. C. (2024). A GPT-based EHR modeling system for unsupervised novel disease detection. Journal of Biomedical Informatics, 157, 104706.
31. Jogi, N. L., Kumar, K. R., Mangala, N., M.Govindarajan, Gamini, P., & Bundhe, S. (2026). Machine Learning-Based Random Forest Model for Sustainable Supply Chain Sales Forecasting. In 2026 6th International Conference on Intelligent Technologies (CONIT) (pp. 1–6). IEEE. 2026 6th International Conference on Intelligent Technologies (CONIT). https://doi.org/10.1109/conit69683.2026.11620788
32. Liu, T., Krentz, A., Lu, L., & Curcin, V. (2024). Machine learning based prediction models for cardiovascular disease risk using electronic health records data: Systematic review and meta-analysis. European Heart Journal—Digital Health, 6(1), 7–22.
33. Kauffman, J., Miotto, R., Klang, E., Costa, A., Norgeot, B., et al. (2025). Embedding methods for electronic health record research. Annual Review of Biomedical Data Science, 8, 563–590.
34. Moglia, V., Johnson, O., Cook, G., de Kamps, M., & Smith, L. (2025). Artificial intelligence methods applied to longitudinal data from electronic health records for prediction of cancer: A scoping review. BMC Medical Research Methodology, 25(1), 24.
