Alcohol Use Prior to Surgery Detected Using AI
Overview
This study focuses on the crucial aspect of preoperative risky alcohol use as a common surgical risk factor and explores the potential of Artificial Intelligence (AI) through natural language processing (NLP) to enhance the identification of such risks from patients’ electronic health records (EHR) before surgery. Examining clinical notes from 53,629 preoperative patients in a tertiary care facility, the research employs a rule-based NLP model for analysis. Additionally, the study compares the effectiveness of NLP with alcohol-related International Classification of Diseases (ICD) diagnosis codes.
The findings reveal that NLP accurately identified 87% of patients labeled with risky alcohol use by human experts, surpassing the performance of diagnosis codes alone, which correctly identified only 29% of such patients.
In terms of specificity, NLP demonstrated a correct identification rate of 84% for the non-risky cohort, while diagnosis codes achieved 90% accuracy.
Notably, in the analysis of the entire dataset, the NLP-based approach identified three times more patients with risky alcohol use compared to ICD codes.
Underscores the efficiency and accuracy of NLP, an AI-based approach, in identifying alcohol-related risks in patients’ EHRs.
This innovative method proves valuable in supplementing existing alcohol screening tools, providing a comprehensive means to identify individuals requiring intervention, treatment, and/or postoperative withdrawal prophylaxis.
The study highlights the limitations of alcohol-related ICD diagnosis codes in comparison to the richer information extraction capabilities of NLP from clinical notes.
Introduction
The study underscores the critical importance of addressing the pervasive issue of risky alcohol use among preoperative patients, emphasizing its status as one of the most common surgical risk factors.
Risky alcohol use, defined as the consumption of more than two standard drinks per day before surgery, has become increasingly prevalent over time.
The consequences of such alcohol habits are profound, leading to heightened risks of infections, wound complications, pulmonary issues, and prolonged hospital stays following elective surgical procedures.
Recognizing the substantial healthcare burden imposed by patients with risky alcohol use and alcohol use disorders, the study proposes an innovative approach to enhance the identification of at-risk individuals.
Despite the considerable impact of alcohol-related complications on patient outcomes and healthcare costs, current screening practices fall short, often due to the reliance on flawed methods, such as single-item questions or untimely assessments.
In response to these limitations, the study advocates for leveraging Electronic Health Records (EHRs) as a rich source of real-world health data.
While acknowledging that alcohol-related conditions are frequently underdiagnosed within structured data like International Classification of Diseases (ICD) codes, the study highlights the untapped potential of unstructured clinical text within EHRs.
This text includes comprehensive information about patients’ alcohol use history, screening data, and details of alcohol-related events, diagnoses, and billing codes.
To harness the wealth of information embedded in clinical text, the study introduces Natural Language Processing (NLP) as a powerful tool within the realm of artificial intelligence.
NLP, a subfield of AI, is designed to derive meaning from human language and text. The study positions NLP as a game-changer in the identification of risky alcohol use, surpassing the limitations of traditional screening methods.
The ability of NLP to understand context, extract nuanced information, and process large volumes of clinical text makes it an ideal candidate for addressing the complex and multifaceted nature of alcohol-related risks.
What sets this study apart is its pioneering effort to develop an NLP-based algorithm explicitly tailored to the surgical context. Surgery presents unique challenges, requiring the synthesis of classification labels across multiple encounters and contexts.
Unlike previous NLP classifiers focused on specific clinics or time-limited sets of notes, this algorithm considers preoperative notes and historical information spanning three years before surgery.
This comprehensive approach allows for a more nuanced and context-aware classification of risky alcohol use among surgical patients.
The study opens new frontiers in the realm of alcohol risk identification by introducing an innovative NLP-based algorithm.
By capitalizing on the extensive information within EHRs, this approach aims to revolutionize the identification of risky alcohol use among preoperative patients.
The potential impact extends beyond mere identification, with the goal of enabling timely interventions, referrals, and alcohol withdrawal prophylaxis when needed.
Ultimately, this research contributes to the broader mission of improving patient outcomes and reducing healthcare costs in the surgical care domain.
Methods
The study conducted an extensive observational cohort investigation within the Michigan Genomic Initiative (MGI) to address the critical issue of classifying pre-surgical patients based on their alcohol consumption patterns.
MGI, a substantial longitudinal cohort study embedded in a prominent Midwestern academic health system, enrolled a substantial number of adult participants who were scheduled to undergo surgery.
The cohort, consisting of 61,502 patients at the time of the study, provided a rich source of longitudinal electronic health information.
The cohort for analysis was meticulously chosen, including patients enrolled between May 29, 2012, and April 17, 2019, who subsequently underwent surgery within 90 days of MGI enrollment, resulting in a refined group of 53,949 patients.
To glean meaningful insights into the patients’ health records, the researchers adeptly extracted text-based clinical records from the past three years, skillfully concatenating notes in chronological order for each patient.
This meticulous curation resulted in a comprehensive and detailed dataset that became the foundation for the subsequent analytical phases.
The primary thrust of the research was to develop a sophisticated natural language processing (NLP) model capable of effectively classifying patients into two distinct categories: those exhibiting “risky alcohol use,” which also encompassed alcohol use disorders, and those who did not display such patterns.
The criteria for defining risky alcohol use were multifaceted, including adherence to national health guidelines, the application of AUDIT-C cut-offs, identification of binge drinking episodes, and recognition of various textual indicators embedded within clinical notes.
The prototype development phase was a hallmark of the study, employing a keyword-driven approach that harnessed the power of 36 meticulously chosen alcohol-related keywords derived from the Unified Medical Language System.
The NLP model’s architecture was designed with precision, encompassing crucial steps such as patient record segmentation, hotspot identification, negation detection, template text removal, patient sex determination, numeric analysis, family history consideration, and an in-depth examination of the patient’s historical records.
The rigorous evaluation of the NLP model’s performance necessitated the creation of a meticulously annotated dataset. Leveraging 1,200 patient records, randomly selected from the cohort, this dataset underwent a meticulous annotation process by four expert annotators.
A subset of 500 patient records served as the training set for the model, while an additional 100 records were earmarked as a blind-labeled test set to rigorously assess the model’s generalization capabilities.
The evaluation metrics employed, including sensitivity, specificity, positive predictive value (PPV), and F1 score, underscored the model’s accuracy and reliability.
The study’s innovative approach wasn’t limited to the NLP model alone; it also conducted a comparative analysis against a baseline labeling methodology reliant on alcohol-related diagnostic codes.
This dual-pronged evaluation aimed to provide a comprehensive understanding of the strengths and limitations of the proposed NLP-driven approach relative to the conventional diagnostic code-based labeling.
In essence, this study stands as a beacon of methodological rigor, utilizing advanced computational techniques to navigate the nuanced landscape of identifying risky alcohol use among pre-surgical patients.
The intricate interplay between clinical expertise, data curation, and computational modeling positions this research at the forefront of endeavors seeking to enhance our understanding of alcohol consumption patterns within healthcare settings.
Results
The study delves into patient and data characteristics, providing a comprehensive overview of the demographic composition and key features of the dataset. Among the participants, 52.6% were female, predominantly White (90.1%), and Non-Hispanic (98.1%).
The mean age of the sample was 53.6 years, with 4.8% having diagnostic codes indicative of risky alcohol use or alcohol use disorder in the past 3 years.
In evaluating the performance of the natural language processing (NLP) approach against the ICD code-based method, the study employed a test evaluation dataset.
Human experts labeled 31 cases as positive for risky alcohol use and 69 as negative. The NLP algorithm identified 38 positive cases, correctly labeling 27, while the ICD codes identified only 16 positive cases.
Notably, the NLP-based approach demonstrated a sensitivity of 0.87, specificity of 0.84, PPV of 0.71, and an F1 score of 0.78, outperforming the ICD code-based approach with a sensitivity of 0.29 and an F1 score of 0.38.
A detailed comparison revealed that, of the 31 positive cases, NLP detected 18 cases missed by ICD codes, while ICD codes failed to identify any cases overlooked by NLP.
When extended to the entire dataset of 53,629 patients, NLP classified 7794 patients as positive for risky alcohol use, while ICD codes identified only 2595.
The agreement between NLP and ICD codes was observed in the positive classification of 1670 patients, with NLP contributing 6124 additional positive cases.
Overall, NLP classified approximately 14.5% of patients as positive, outperforming the 4.8% positive classifications by ICD codes.
Conclusion
The discussion emphasizes the potential of natural language processing, an artificial intelligence-based method, as a promising and accurate tool for identifying patients with risky alcohol use and alcohol use disorders from electronic health records (EHRs).
The traditional reliance on diagnostic codes or manual chart review is compared to the NLP-based approach, showcasing the latter’s superiority in accuracy.
The study concludes that NLP can significantly enhance the identification of patients with alcohol-related issues, particularly those at an increased risk of surgical complications, highlighting the limitations of relying solely on diagnostic codes.
Oncology Related Tools
Other
Latest Research
