In August of 2019, the Pittsburgh Health Data Alliance (PHDA) and Amazon Web Services (AWS) announced a new collaboration to advance innovation in areas such as cancer diagnostics, precision medicine, electronic health records, and medical imaging.
Researchers from the University of Pittsburgh Medical Center (UPMC), the University of Pittsburgh, and Carnegie Mellon University (CMU), who were already supported by the PHDA, received additional support from Amazon Research Awards to use machine learning techniques to study breast cancer risk, identify depression markers, and understand what drives tumor growth, among other projects.
In work funded through the PHDA-AWS collaboration, a research team led by Shandong Wu, an associate professor in the University of Pittsburgh Department of Radiology, is using deep-learning systems to analyze mammograms in order to predict the short‐term risk of developing breast cancer. A team of experts in computer vision, deep learning, bioinformatics, and breast cancer imaging are working together to develop a more personalized approach for patients undergoing breast cancer screening.
Wu and his colleagues collected 452 de-identified normal screening mammogram images from 226 patients, half of whom later developed breast cancer and half of whom did not. Leveraging AWS tools, such as Amazon SageMaker, they used two different machine learning models to analyze the images for characteristics that could help predict breast cancer risk. As they reported in the American Association of Physicists in Medicine, both models consistently outperformed the simple measure of breast density, which today is the primary imaging marker for breast cancer risk. The team’s models demonstrated between 33% and 35% improvement over these existing models, based on metrics that incorporate sensitivity and specificity.
“This preliminary work demonstrates the feasibility and promise of applying deep-learning methodologies for in-depth interpretation of mammogram images to enhance breast cancer risk assessment,” Wu said. “Identifying additional risk factors for breast cancer, including those that can lead to a more personalized approach to screening, may help patients and providers take more appropriate preventive measures to reduce the likelihood of developing the disease or catching it early on when interventions are most effective. “
Tools that could provide more accurate predictions from screening images could be used to guide clinical decision making related to frequency of follow-up imaging and other forms of preventative monitoring. This could reduce unnecessary imaging examinations or clinical procedures, decreasing patients’ anxiety resulting from inaccurate risk assessments, and cutting costs.
Moving forward, researchers at the University of Pittsburgh and UPMC will pursue studies with more training samples and longitudinal imaging data to further evaluate the models. They also plan to combine deep learning with known clinical risk factors to improve upon the ability to diagnose and treat breast cancer earlier.
In a second project, Louis-Philippe Morency, associate professor of computer science at CMU, and Eva Szigethy, a clinical researcher at UPMC and professor of psychiatry, medicine, and pediatrics at the University of Pittsburgh, are developing sensing technologies that can automatically measure subtle changes in individuals’ behavior — such as facial expressions and use of language — that can act as biomarkers for depression.
These biomarkers will later be compared with the results of traditional clinical assessments, allowing investigators to evaluate the performance of their technology and make improvements where necessary. This machine learning technology is intended to complement the ability of a clinician to make decisions about diagnosis and treatment. The team is working with a gastrointestinal-disorder clinic at UPMC, due to the high rate of depression observed in patients with functional gastrointestinal disorders.
This work involves training machine learning models on tens of thousands of examples across multiple modalities, including language (the spoken word), acoustic (prosody), and visual (facial expressions). The computational load is heavy, but by running experiments in parallel on multiple GPUS AWS services have allowed the researchers to train their models in a few days instead of weeks.
A quick and objective marker of depression could help clinicians more efficiently assess patients at baseline, identify patients who would otherwise go undiagnosed, and more accurately measure patients’ responses to interventions. The team presented a paper on the work, “Integrating Multimodal Information in Large Pretrained Transformers”, at the July 2020 meeting of the Association for Computational Linguistics.
“Depression is a disease that affects more than 17 million adults in the United States, up to two-thirds of all depression cases are left undiagnosed and therefore untreated,” Morency said. “New insights to increase the accuracy, efficiency, and adoption of depression screening have the potential to impact millions of patients, their families, and the healthcare system as a whole.”
The research projects on breast cancer and depression represent just the tip of the iceberg when it comes to the research and insights the collaboration across PHDA and AWS will ultimately deliver to improve patient care. Teams of researchers, health-care professionals, and machine learning experts across the PHDA continue to make progress on key research topics, from the risk of aneurysms and predicting how cancer cells progress, to improving the complex electronic-health-records system.
”Amazon is excited and encouraged by the progress these researchers are making and how machine learning is central to their work,” says An Luo, senior technical program manager for academic programs, Amazon AI. “We look forward to continuing to share how this unique collaboration between the PHDA and AWS is enabling new discoveries to help patients on a global scale.”
