VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models

Machine learning (ML) is increasingly applied to Electronic Health Records (EHRs) to solve clinical prediction tasks. Although many ML models perform promisingly, issues with model transparency and interpretability limit their adoption in clinical practice. Directly using existing explainable ML techniques in clinical settings can be challenging. Through literature surveys and collaborations with six clinicians with an average of 17 years of clinical experience, we identified three key challenges, including clinicians' unfamiliarity with ML features, lack of contextual information, and the need for cohort-level evidence. Following an iterative design process, we further designed and developed VBridge, a visual analytics tool that seamlessly incorporates ML explanations into clinicians' decision-making workflow. The system includes a novel hierarchical display of contribution-based feature explanations and enriched interactions that connect the dots between ML features, explanations, and data. We demonstrated the effectiveness of VBridge through two case studies and expert interviews with four clinicians, showing that visually associating model explanations with patients' situational records can help clinicians better interpret and use model predictions when making clinician decisions. We further derived a list of design implications for developing future explainable ML tools to support clinical decision-making.

[1]  Jimeng Sun,et al.  CarePre , 2018, ACM Trans. Comput. Heal..

[2]  Adrin Jalali,et al.  Interpretable per case weighted ensemble method for cancer associations , 2014, bioRxiv.

[3]  S. Brunak,et al.  Mining electronic health records: towards better research applications and clinical care , 2012, Nature Reviews Genetics.

[4]  Vivian West,et al.  Innovative information visualization of electronic health record data: a systematic review , 2014, J. Am. Medical Informatics Assoc..

[5]  Jiang Bian,et al.  Explainable artificial intelligence models using real-world electronic health record data: a systematic scoping review , 2020, J. Am. Medical Informatics Assoc..

[6]  Aram Galstyan,et al.  Multitask learning and benchmarking with clinical time series data , 2017, Scientific Data.

[7]  Daniel A. Keim,et al.  CloudLines: Compact Display of Event Episodes in Multiple Time-Series , 2011, IEEE Transactions on Visualization and Computer Graphics.

[8]  David Gotz,et al.  DecisionFlow: Visual Analytics for High-Dimensional Temporal Event Sequence Data , 2014, IEEE Transactions on Visualization and Computer Graphics.

[9]  Yixin Chen,et al.  ECGLens: Interactive Visual Exploration of Large Scale ECG Data for Arrhythmia Detection , 2018, CHI.

[10]  Huamin Qu,et al.  RuleMatrix: Visualizing and Understanding Classifiers with Rules , 2018, IEEE Transactions on Visualization and Computer Graphics.

[11]  Enrico Bertini,et al.  INFUSE: Interactive Feature Selection for Predictive Modeling of High Dimensional Data , 2014, IEEE Transactions on Visualization and Computer Graphics.

[12]  Ben Shneiderman,et al.  Interactive Information Visualization to Explore and Query Electronic Health Records , 2013, Found. Trends Hum. Comput. Interact..

[13]  Jeffrey Dean,et al.  Scalable and accurate deep learning with electronic health records , 2018, npj Digital Medicine.

[14]  Ben Shneiderman,et al.  LifeLines: using visualization to enhance navigation and analysis of patient records , 1998, AMIA.

[15]  Roy A. Ruddle,et al.  Using Dashboard Networks to Visualize Multiple Patient Histories: A Design Study on Post-Operative Prostate Cancer , 2019, IEEE Transactions on Visualization and Computer Graphics.

[16]  Yu-Ru Lin,et al.  Voila: Visual Anomaly Detection and Monitoring with Streaming Spatiotemporal Data , 2018, IEEE Transactions on Visualization and Computer Graphics.

[17]  Li Li,et al.  Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records , 2016, Scientific Reports.

[18]  Maneesh Agrawala,et al.  Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Charts , 2021, CHI.

[19]  Jimeng Sun,et al.  DICON: Interactive Visual Analysis of Multidimensional Clusters , 2011, IEEE Transactions on Visualization and Computer Graphics.

[20]  Yan Liu,et al.  Interpretable Deep Models for ICU Outcome Prediction , 2016, AMIA.

[21]  Valentino Constantinou,et al.  Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding , 2018, KDD.

[22]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[23]  Kalyan Veeramachaneni,et al.  Deep feature synthesis: Towards automating data science endeavors , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[24]  K. Borgwardt,et al.  Machine Learning in Medicine , 2015, Mach. Learn. under Resour. Constraints Vol. 3.

[25]  Wendong Ge,et al.  An Interpretable ICU Mortality Prediction Model Based on Logistic Regression and Recurrent Neural Networks with LSTM Units , 2018, AMIA.

[26]  Mingliang Xu,et al.  Towards Better Bus Networks: A Visual Analytics Approach , 2021, IEEE Transactions on Visualization and Computer Graphics.

[27]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[28]  Jesse Johnson,et al.  ClinicalVis: Supporting Clinical Task-Focused Design Evaluation , 2018, ArXiv.

[29]  Ben Shneiderman,et al.  Cohort Comparison of Event Sequences with Balanced Integration of Visual Analytics and Statistics , 2015, IUI.

[30]  Huamin Qu,et al.  DECE: Decision Explorer with Counterfactual Explanations for Machine Learning Models , 2020, IEEE Transactions on Visualization and Computer Graphics.

[31]  Max Welling,et al.  Interpretation of microbiota-based diagnostics by explaining individual classifier decisions , 2017, BMC Bioinformatics.

[32]  Ben Shneiderman,et al.  Finding comparable temporal categorical records: A similarity measure with an interactive visualization , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[33]  John Zimmerman,et al.  Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes , 2019, CHI.

[34]  Scott M. Lundberg,et al.  Explainable machine-learning predictions for the prevention of hypoxaemia during surgery , 2018, Nature Biomedical Engineering.

[35]  L. Shapley A Value for n-person Games , 1988 .

[36]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[37]  Suzette J. Bielinski,et al.  Use of diverse electronic medical record systems to identify genetic risk for type 2 diabetes within a genome-wide association study , 2012, J. Am. Medical Informatics Assoc..

[38]  Ben Shneiderman,et al.  Querying event sequences by exact match or similarity search: Design and empirical evaluation , 2012, Interact. Comput..

[39]  Tamara Munzner,et al.  Design Study Methodology: Reflections from the Trenches and the Stacks , 2012, IEEE Transactions on Visualization and Computer Graphics.

[40]  Qian Yang,et al.  Designing Theory-Driven User-Centric Explainable AI , 2019, CHI.

[41]  Jan Robinson Facm Evidence-based medicine how to practice & teach EBM , 1997 .

[42]  Marc Streit,et al.  A visual approach to explainable computerized clinical decision support , 2020, Comput. Graph..

[43]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[44]  Anna Goldenberg,et al.  What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use , 2019, MLHC.

[45]  Stephanie Guerlain,et al.  Evaluating the Use of Flowsheets in Pediatric Intensive Care to Inform Design , 2006 .

[46]  Yuval Shahar,et al.  Distributed, intelligent, interactive visualization and exploration of time-oriented clinical data and their abstractions , 2006, Artif. Intell. Medicine.

[47]  Krist Wongsuphasawat,et al.  Outflow : Visualizing Patient Flow by Symptoms and Outcome , 2011 .

[48]  Kalyan Veeramachaneni,et al.  Cardea: An Open Automated Machine Learning Framework for Electronic Health Records , 2020, 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA).

[49]  Jimeng Sun,et al.  Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review , 2018, J. Am. Medical Informatics Assoc..

[50]  David C. Kale,et al.  Sim•TwentyFive: An Interactive Visualization System for Data-Driven Decision Support , 2012, AMIA.

[51]  Jürgen Bernard,et al.  A Visual-Interactive System for Prostate Cancer Cohort Analysis , 2015, IEEE Computer Graphics and Applications.

[52]  Ankur Teredesai,et al.  Interpretable Machine Learning in Healthcare , 2018, 2018 IEEE International Conference on Healthcare Informatics (ICHI).

[53]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[54]  Fei Wang,et al.  A methodology for interactive mining and visual analysis of clinical event patterns using electronic health record data , 2014, J. Biomed. Informatics.

[55]  Huilong Duan,et al.  PIC, a paediatric-specific intensive care database , 2020, Scientific Data.

[56]  Martin Wattenberg,et al.  The What-If Tool: Interactive Probing of Machine Learning Models , 2019, IEEE Transactions on Visualization and Computer Graphics.

[57]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[58]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[59]  Jimeng Sun,et al.  RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records , 2018, IEEE Transactions on Visualization and Computer Graphics.

[60]  Zachary C. Lipton,et al.  The Doctor Just Won't Accept That! , 2017, 1711.08037.

[61]  A. Fleisher,et al.  Clinical predictors of progression to Alzheimer disease in amnestic mild cognitive impairment , 2007, Neurology.

[62]  Yuval Shahar,et al.  Intelligent visualization and exploration of time-oriented clinical data , 1999, Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers.

[63]  Shamim Nemati,et al.  An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU , 2017, Critical care medicine.

[64]  Michael Gao,et al.  "The human body is a black box": supporting clinical decision-making with deep learning , 2019, FAT*.

[65]  Prabhakar M. Dixit,et al.  Visual analytics for evaluating clinical pathways , 2017, 2017 IEEE Workshop on Visual Analytics in Healthcare (VAHC).

[66]  Bernhard Preim,et al.  Subpopulation Discovery and Validation in Epidemiological Data , 2017, EuroVA@EuroVis.

[67]  Kenney Ng,et al.  Clustervision: Visual Supervision of Unsupervised Clustering , 2018, IEEE Transactions on Visualization and Computer Graphics.