Using Deep Learning with Canadian Primary Care Data for Disease Diagnosis

The majority of Canadian primary care systems record patient data in the form of Electronic Medical Records (EMR). EMRs hold structured, semi-structured and unstructured demographic and health care data about patients. The value of EMR data for research, health surveillance and quality improvement continues to be explored. Data analytics such as Machine Learning (ML) and statistical modeling techniques have been applied to de-identified EMR data repositories to advance our understanding of different health conditions and patient care. More recently, the application of Deep Learning (DL) approaches to structured, semi-structured and unstructured data of the EMRs is being investigated as an avenue for improved identification of health conditions. Supervised ML methods have dominated disease classification for more prevalent diseases. A large cohort of labeled data is required to train ML models using supervised learning methods. For less common diseases, the amount of available labeled data is often insufficient, and a variety of strategies are being explored to deal with inadequate, noisy and missing data. This chapter describes the benefits of using DL models with EMR data for research to improve provisioning of health care in primary care settings. A few prominent DL models such as Multi-Layered Perceptron (MLP), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) are discussed with example scenarios that demonstrate application of some of these predictive analytics models to both structured and unstructured EMR data using regular and weak supervision methods for diagnosing both prevalent and non-prevalent diseases.

[1]  Xiaojin Zhu,et al.  Semi-Supervised Learning Literature Survey , 2005 .

[2]  Robert J. Marks,et al.  Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks , 1999 .

[3]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[4]  Svetha Venkatesh,et al.  $\mathtt {Deepr}$: A Convolutional Net for Medical Records , 2016, IEEE Journal of Biomedical and Health Informatics.

[5]  R. Birtwhistle,et al.  Canadian Primary Care Sentinel Surveillance Network: a developing resource for family medicine and public health. , 2011, Canadian family physician Medecin de famille canadien.

[6]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[7]  Mark L. Braunstein Patient — Physician collaboration on FHIR (Fast Healthcare Interoperability Resources) , 2015, 2015 International Conference on Collaboration Technologies and Systems (CTS).

[8]  N. Tangri,et al.  Prevalence and Demographics of CKD in Canadian Primary Care Practices: A Cross-sectional Study , 2019, Kidney international reports.

[9]  Lior Wolf,et al.  In Defense of Word Embedding for Generic Text Representation , 2015, NLDB.

[10]  Takumi Kobayashi,et al.  Patient Stress Estimation for Using Deep Learning with RRI Data Sensed by WBAN , 2019, 2019 13th International Symposium on Medical Information and Communication Technology (ISMICT).

[11]  Tyler Williamson,et al.  Validating the 8 CPCSSN Case Definitions for Chronic Disease Surveillance in a Primary Care Database of Electronic Health Records , 2014, The Annals of Family Medicine.

[12]  Casimir A. Kulikowski,et al.  Representation of Expert Knowledge for Consultation: The CASNET and EXPERT Projects , 2019, Artificial Intelligence in Medicine.

[13]  J. Myers,et al.  The INTERNIST-1/QUICK MEDICAL REFERENCE project--status report. , 1986, The Western journal of medicine.

[14]  Christopher Ré,et al.  Snorkel: Rapid Training Data Creation with Weak Supervision , 2017, Proc. VLDB Endow..

[15]  J. Marc Overhage,et al.  Effectiveness of LSTMs in Predicting Congestive Heart Failure Onset , 2019, ArXiv.

[16]  Truyen Tran,et al.  Predicting healthcare trajectories from medical records: A deep learning approach , 2017, J. Biomed. Informatics.

[17]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[18]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[19]  Sunghwan Sohn,et al.  Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications , 2010, J. Am. Medical Informatics Assoc..

[20]  Alexander Singer,et al.  Identifying people with multiple sclerosis in the Canadian Primary Care Sentinel Surveillance Network , 2019, Multiple sclerosis journal - experimental, translational and clinical.

[21]  Li Li,et al.  Using LSTM and GRU neural network methods for traffic flow prediction , 2016, 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC).

[22]  Dongyan Zhao,et al.  Learning to Predict Charges for Criminal Cases with Legal Basis , 2017, EMNLP.

[23]  Zachariah Zhang,et al.  Deep EHR: Chronic Disease Prediction Using Medical Notes , 2018, MLHC.

[24]  E H Shortliffe,et al.  Medical expert systems--knowledge tools for physicians. , 1986, The Western journal of medicine.

[25]  Svetha Venkatesh,et al.  DeepCare: A Deep Dynamic Memory Model for Predictive Medicine , 2016, PAKDD.

[26]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[27]  Divya Tomar,et al.  A survey on Data Mining approaches for Healthcare , 2013, BSBT 2013.

[28]  Rajesh Ranganath,et al.  ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission , 2019, ArXiv.

[29]  A. Singer,et al.  Enhancing Health Surveillance: Validation of a Novel Electronic Medical Records-Based Definition of Cases of Pediatric Type 1 and Type 2 Diabetes Mellitus. , 2019, Canadian journal of diabetes.

[30]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[31]  David A. Ferrucci,et al.  UIMA: an architectural approach to unstructured information processing in the corporate research environment , 2004, Natural Language Engineering.

[32]  Olivier Bodenreider,et al.  The Unified Medical Language System (UMLS): integrating biomedical terminology , 2004, Nucleic Acids Res..

[33]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[34]  Preslav Nakov,et al.  SemEval-2013 Task 2: Sentiment Analysis in Twitter , 2013, *SEMEVAL.

[35]  Jinfeng Yang,et al.  Clinical Relation Extraction with Deep Learning , 2016 .

[36]  Zhiyuan Liu,et al.  Legal Judgment Prediction via Topological Learning , 2018, EMNLP.

[37]  Mihhail Matskin,et al.  Deep Text Mining of Instagram Data without Strong Supervision , 2018, 2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[38]  Kai Chen,et al.  A LSTM-based method for stock returns prediction: A case study of China stock market , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[39]  David Barber,et al.  Detecting Low Back Pain from Clinical Narratives Using Machine Learning Approaches , 2018, DEXA Workshops.

[40]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[41]  Hong Yu,et al.  Bidirectional RNN for Medical Event Detection in Electronic Health Records , 2016, NAACL.

[42]  Zhiyong Lu,et al.  TaggerOne: joint named entity recognition and normalization with semi-Markov Models , 2016, Bioinform..

[43]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[44]  Parisa Rashidi,et al.  Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis , 2017, IEEE Journal of Biomedical and Health Informatics.

[45]  Giorgio Maria Di Nunzio,et al.  A Linguistic Failure Analysis of Classification of Medical Publications: A Study on Stemming vs Lemmatization , 2018, CLiC-it.

[46]  Florian Metze,et al.  Audio-based multimedia event detection using deep recurrent neural networks , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[47]  David Barber,et al.  Using machine learning to predict hypertension from a clinical dataset , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[48]  Smaranda Belciug,et al.  Error-correction learning for artificial neural networks using the Bayesian paradigm. Application to automated medical diagnosis , 2014, J. Biomed. Informatics.

[49]  N. Drummond,et al.  Representativeness of patients and providers in the Canadian Primary Care Sentinel Surveillance Network: a cross-sectional study. , 2016, CMAJ open.

[50]  James H. Harrison,et al.  Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record , 2018, IEEE Access.

[51]  Jimeng Sun,et al.  Explainable Prediction of Medical Codes from Clinical Text , 2018, NAACL.

[52]  Jimeng Sun,et al.  Multi-layer Representation Learning for Medical Concepts , 2016, KDD.

[53]  Omer Levy,et al.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding , 2018, BlackboxNLP@EMNLP.

[54]  Franck Dernoncourt,et al.  De-identification of patient notes with recurrent neural networks , 2016, J. Am. Medical Informatics Assoc..

[55]  R. Birtwhistle,et al.  The Prevalence and Risk for Herpes Zoster Infection in Adult Patients With Diabetes Mellitus in the Canadian Primary Care Sentinel Surveillance Network. , 2017, Canadian journal of diabetes.

[56]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[57]  M. Kramer Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[58]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[59]  Liang Zhao,et al.  SimNest: Social Media Nested Epidemic Simulation via Online Semi-Supervised Deep Learning , 2015, 2015 IEEE International Conference on Data Mining.

[60]  Jaewoo Kang,et al.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining , 2019, Bioinform..

[61]  Alan R. Aronson,et al.  An overview of MetaMap: historical perspective and recent advances , 2010, J. Am. Medical Informatics Assoc..

[62]  Li Li,et al.  Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records , 2016, Scientific Reports.

[63]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[64]  Zhiyuan Liu,et al.  Few-Shot Charge Prediction with Discriminative Legal Attributes , 2018, COLING.

[65]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[66]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[67]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[68]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[69]  T. Williamson,et al.  From patient care to research: a validation study examining the factors contributing to data quality in a primary care electronic medical record database , 2015, BMC Family Practice.

[70]  Alain Moren,et al.  A large outbreak of hepatitis E among a displaced population in Darfur, Sudan, 2004: the role of water treatment methods. , 2006, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[71]  A. Singer,et al.  Characterizing patients with high use of the primary and tertiary care systems: A retrospective cohort study. , 2020, Health policy.

[72]  Hongfang Liu,et al.  Journal of Biomedical Informatics , 2022 .

[73]  Noémie Elhadad,et al.  Multi-Label Classification of Patient Notes: Case Study on ICD Code Assignment , 2018, AAAI Workshops.

[74]  Bhuvana Ramabhadran,et al.  Bag-of-word normalized n-gram models , 2008, INTERSPEECH.

[75]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[76]  Jason H. Moore,et al.  Mapping Patient Trajectories using Longitudinal Extraction and Deep Learning in the MIMIC-III Critical Care Database , 2017, bioRxiv.

[77]  G. Gorry,et al.  Towards the simulation of clinical cognition. Taking a present illness by computer. , 1976, The American journal of medicine.

[78]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[79]  Tina Hernandez-Boussard,et al.  Weakly supervised natural language processing for assessing patient-centered outcome following prostate cancer treatment , 2019, JAMIA open.

[80]  Koji Zettsu,et al.  Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting PM2.5 , 2015, Neural Computing and Applications.

[81]  Kyle Lo,et al.  SciBERT: Pretrained Contextualized Embeddings for Scientific Text , 2019, ArXiv.

[82]  Y. Elbert,et al.  Disease outbreak detection system using syndromic data in the greater Washington DC area. , 2002, American journal of preventive medicine.

[83]  Shang Gao,et al.  Hierarchical attention networks for information extraction from cancer pathology reports , 2017, J. Am. Medical Informatics Assoc..

[84]  Hongfang Liu,et al.  A clinical text classification paradigm using weak supervision and deep representation , 2019, BMC Medical Informatics and Decision Making.

[85]  P. Lakhani,et al.  Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks. , 2017, Radiology.

[86]  Christopher De Sa,et al.  Data Programming: Creating Large Training Sets, Quickly , 2016, NIPS.

[87]  Jing Xiao,et al.  EasiCSDeep: A deep learning model for Cervical Spondylosis Identification using surface electromyography signal , 2018, ArXiv.

[88]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[89]  Zahir Tari,et al.  A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis , 2014, IEEE Transactions on Emerging Topics in Computing.

[90]  Meng Wang,et al.  Disease Inference from Health-Related Questions via Sparse Deep Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[91]  Farhana Zulkernine,et al.  Predicting Chronic Obstructive Pulmonary Disease from EMR data , 2020, 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB).

[92]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[93]  Liang Xiao,et al.  Developing a rule-driven clinical decision support system with an extensive and adaptative architecture , 2012, 2012 IEEE 14th International Conference on e-Health Networking, Applications and Services (Healthcom).

[94]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[95]  N. Drummond,et al.  Development of a validated algorithm for the diagnosis of paediatric asthma in electronic medical records , 2016, npj Primary Care Respiratory Medicine.

[96]  Hong Zhu,et al.  A survey on feature extraction for pattern recognition , 2011, Artificial Intelligence Review.

[97]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[98]  K. Collins,et al.  Using Electronic Medical Record to Identify Patients With Dyslipidemia in Primary Care Settings: International Classification of Disease Code Matters From One Region to a National Database , 2017, Biomedical informatics insights.

[99]  Jimeng Sun,et al.  Using recurrent neural network models for early detection of heart failure onset , 2016, J. Am. Medical Informatics Assoc..

[100]  Karim Keshavjee,et al.  Ethics and privacy issues of a practice-based surveillance system: need for a national-level institutional research ethics board and consent standards. , 2011, Canadian family physician Medecin de famille canadien.

[101]  Michel Dojat,et al.  Application of Information Technology: A UMLS-based Knowledge Acquisition Tool for Rule-based Clinical Decision Support System Development , 2001, J. Am. Medical Informatics Assoc..

[102]  M. Meade,et al.  Critically ill patients with 2009 influenza A(H1N1) infection in Canada. , 2009, JAMA.

[103]  Keechul Jung,et al.  GPU implementation of neural networks , 2004, Pattern Recognit..

[104]  Ryen W. White,et al.  An Insight Extraction System on BioMedical Literature with Deep Neural Networks , 2017, EMNLP.

[105]  Umang Bhatt,et al.  Diagnostic Model Explanations: A Medical Narrative , 2019, AAAI Spring Symposium: Interpretable AI for Well-being.

[106]  Patrice Y. Simard,et al.  High Performance Convolutional Neural Networks for Document Processing , 2006 .

[107]  Chiou-Shann Fuh,et al.  A Rule-Based Clinical Decision Model to Support Interpretation of Multiple Data in Health Examinations , 2011, Journal of Medical Systems.

[108]  F. Chang,et al.  Progress in electronic medical record adoption in Canada. , 2015, Canadian family physician Medecin de famille canadien.

[109]  Alessandro Tredicucci,et al.  Corrigendum: Universal lineshapes at the crossover between weak and strong critical coupling in Fano-resonant coupled oscillators , 2016, Scientific Reports.

[110]  Farhana Zulkernine,et al.  Diagnosing PTSD using electronic medical records from canadian primary care data , 2019, NSysS.

[111]  Shamim Nemati,et al.  Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).