TR: What educational programs and training did you seek out to become skilled in the application of bioinformatics and artificial intelligence to research activities?
Dr. Park: Luckily, as part of my research track, I completed a master’s degree in patient-oriented research at the Mailman School of Public Health. I was able to enroll in a few elective classes that focused on data science, machine learning and bioinformatics. Columbia has a really strong Department of Biomedical Informatics (DBMI), and I was able to enroll in a course that focused on modeling clinical terminologies and vocabularies (like International Classification of Diseases, SNOMED CT and Unified Medical Language System) extracted from EHR data and mapping them into a common language model, which is basically one of the premises of OHDSI.
TR: Briefly, can you explain what a large language model is and how such models can be used to conduct research?
Dr. Park: Large language models (LLMs) are algorithms/tools designed to automatically process and extract from large volumes of text; these models are usually optimized and primed through prior training. Clinical researchers have utilized LLMs to process clinical documents, educational materials or scientific abstracts and manuscripts to extract important clinical variables and characteristics or to synthesize literature.
TR: How smart are these algorithms in accurately identifying relevant information in a patient’s chart?
Dr. Park: I guess this is what remains to be seen. Expectations seem high for clinicians. For instance, we want these models to perform efficiently and accurately across a range of many tasks, including synthesizing complex clinical/patient data like discharge summaries, translating clinical language into patient-centered language summaries, providing appropriate response templates for patient questions on messaging systems, and even suggesting differential diagnoses and treatment strategies (which encroaches into clinical decision making). In the RA landscape, we are using these more and more to correctly identify the diagnosis in huge volumes of text drawn from the EHR, as well as from important clinical elements (like disease activity scores).
I think the biggest issues right now include the fact that LLMs can create hallucinations (i.e., information that is false, fabricated, nonsensical and/or not present in the data elements you presented to the model) and reinforce assumptions and biases inherent in the data you fed it. So those areas need to be fine-tuned to create truly smart LLMs.
TR: What are some of the potential challenges of using LLMs in the type of research you do?
