The system could help doctors select the least risky treatments in urgent situations, such as treating sepsis.
Sepsis kills nearly 270,000 people in the United States each year. The unpredictable medical condition can progress rapidly, leading to rapid drops in blood pressure, tissue damage, multiple organ failure, and death.
Immediate interventions by healthcare professionals save lives, but some sepsis treatments can also contribute to patient deterioration, so choosing the optimal therapy can be a difficult task. For example, in the early morning hours of severe sepsis, giving too much fluid intravenously can increase a patient’s risk of death.
To help doctors avoid drugs that could potentially contribute to a patient’s death, researchers at WITH and elsewhere have developed a machine learning model that could be used to identify treatments that pose a higher risk than other options. Their model can also warn doctors when a septic patient is approaching a medical impasse – the point where the patient is most likely to die no matter what treatment is used – so they can intervene before it’s too late.
When applied to a dataset of sepsis patients in a hospital intensive care unit, the researchers’ model showed that about 12 percent of treatments given to deceased patients were harmful. The study also shows that about 3 percent of patients who did not survive had reached a medical impasse up to 48 hours before death.
“We see that our model is almost eight hours ahead of a doctor’s determination of a patient’s deterioration. This is very important because in these really sensitive situations every minute counts and it is very important to be aware of the patient’s evolution and the risk of a particular treatment at any given time, ”says Taylor Killian, a PhD student at Healthy ML -Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL).
In addition to Killian, his advisor, Assistant Professor Marzyeh Ghassemi, leader of the Healthy ML group and senior author; Lead author Mehdi Fatemi, a senior researcher at Microsoft Research; and Jayakumar Subramanian, Senior Research Scientist at Adobe India. The research results will be presented at this week’s conference on neural information processing systems.
A lack of data
This research project was spurred by a 2019 paper by Fatemi that examined the use of reinforcement learning in situations where it is too dangerous to examine arbitrary actions, making it difficult to generate enough data for algorithms to be effective work out. These situations in which no more data can be proactively collected are known as “offline” settings.
In reinforcement learning, the algorithm is trained through trial and error and learns to take action that will maximize its reward. In healthcare, however, it is almost impossible to generate enough data for these models to learn the optimal treatment because it is not ethical to experiment with possible treatment strategies.
So the researchers turned reinforcement learning on its head. They used the limited data of a hospital intensive care unit to train a reinforcement learning model to identify treatments to be avoided with the aim of preventing a patient from entering a medical impasse.
Learning what to avoid is a more statistically efficient approach that requires less data, explains Killian.
“When we think of dead ends in driving a car, we might think this is the end of the road, but you could probably classify every foot on this road towards the dead end as a dead end. As soon as you turn off another route, you will find yourself at a dead end. This is how we define a medical impasse: Once you’ve embarked on a path in which the patient will walk towards death regardless of their decision, ”says Killian.
“A key idea here is to reduce the likelihood of choosing any treatment relative to the chance of bringing the patient into a medical cul-de-sac – a trait known as treatment safety. This is a difficult problem to solve because the data doesn’t directly give us such insight. Our theoretical results enabled us to transform this core idea into a problem of reinforcement learning, ”says Fatemi.
To develop their approach called Dead-end Discovery (DeD), they made two copies of a neural network. The first neural network focuses only on negative outcomes – if a patient died – and the second network only focuses on positive outcomes – if a patient survived. By using two neural networks separately, the researchers were able to identify a risky treatment in one and confirm it with the other.
They fed each neural network patient health statistics and a suggested treatment. The networks give an estimated value of this treatment and also assess the likelihood that the patient will hit a medical impasse. The researchers compared these estimates to establish thresholds to see if the situation was showing any signs.
A yellow flag means a patient is entering an area of concern, while a red flag means a situation where the patient is very likely not to recover.
Treatment is important
The researchers tested their model on a dataset of patients suspected of being septic in the intensive care unit at Beth Israel Deaconess Medical Center. This dataset contains approximately 19,300 observations over a 72-hour period that focuses on when patients first show symptoms of sepsis. Their results confirmed that some patients on the dataset had hit medical dead ends.
The researchers also found that 20 to 40 percent of patients who did not survive hoisted at least one yellow flag before they died, and many hoisted that flag at least 48 hours before they died. The results also showed that when comparing the trends of survived and deceased patients, once a patient flies their first flag, there is a very large variance in the value of the treatments given. The time window around the first flag is a critical point in treatment decisions.
“This helped us to confirm that treatment is important and that treatment differs from how patients survive and how patients don’t. We found that more than 11 percent of suboptimal treatments could potentially have been avoided because doctors had better alternatives at the time. That’s a pretty respectable number when you consider the number of patients worldwide who were septic at any given point in time in the hospital, ”says Killian.
Ghassemi is also quick to point out that the model is intended to support, not replace, doctors.
“We want human clinicians to make decisions about care, and advice on what treatments to avoid isn’t going to change that,” she says. “We can identify risks and add relevant guard rails based on the results of 19,000 patient treatments – that’s equivalent to a single nurse who sees more than 50 septic patient results every day for an entire year.”
In the future, the researchers also want to assess causal relationships between treatment decisions and the development of patient health. They plan to continue improving the model so that it can produce uncertainty estimates about treatment values that would help clinicians make more informed decisions. Another way to further validate the model is to apply it to data from other hospitals, which will be sought in the future.
Relation: “Medical dead ends and learning to identify high risk conditions and treatments“By Mehdi Fatemi, Taylor W. Killian, Jayakumar Subramanian and Marzyeh Ghassemi, NeurIPS procedure.
This research was supported in part by Microsoft Research, an Azrieli Global Scholar Chair from the Canadian Institute for Advanced Research, a Canada Research Council Chair, and a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada.