HHS CREATE explores new approach to AI machine learning
What if there was an even better, more accurate way to ensure consistent, correct dosages of medications like insulin? The HHS CREATE team explored this using artificial intelligence that teaches itself.
If you know someone with diabetes, you may have noticed them wearing an insulin pump. It may even be a high-tech version, consisting of a continuous glucose monitor and computer program that calculates how much insulin they need to keep blood glucose levels in check and automatically deliver dosages.
It’s advanced technology by today’s standards. But what if there was an even better, more accurate way to ensure consistent, correct dosages in medications like insulin, or blood-thinners warfarin and heparin by using artificial intelligence (AI) that teaches itself?
Dr. Ted Scott, vice president of innovation and partnerships for HHS.
A study by the Hamilton Health Sciences (HHS) CREATE (Centre for Data Science and Digital Health) team explored this, opening the door to use this type of AI in health-care research. “Our results were very promising, and have caught the attention of the wider scientific community,” says Dr. Ted Scott, vice president of innovation and partnerships for HHS.
Medications like insulin, warfarin and heparin need to be dosed just right or they can trigger serious health issues. Warfarin, for example, prevents blood clots, but too small a dose and patients are at risk of a stroke. Too much, and they could start bleeding spontaneously because their blood becomes too thin.
Artificial intelligence that uses reinforcement learning could help advance health-care technology for such patients, says Dr. Jeremy Petch, director of CREATE. Staffed with experts in software engineering, AI and data sciences, CREATE works with hospital clinicians including physicians to develop new ideas and create digital solutions that fundamentally reimagine how health care is delivered. CREATE also works with organizations in both the public and private sector, from local start-ups to national agencies and international bodies.
AI reinforcement learning
With reinforcement learning, AI mimics the trial-and-error process that humans use for learning. Algorithms with a reward-and-punishment pattern learn from mistakes in order to find the best ways to achieve positive outcomes.
“We wanted to explore whether AI could do a better job of helping to get the dose just right on a continual basis,” Dr. Jeremy Petch
Dr. Jeremy Petch, director of CREATE
“It’s the same paradigm of AI that led to computers beating humans in games like chess,” says Petch. “Through trial and error, AI discovers optimal strategies.”
Other types of AI learning are already used in health care, but reinforcement learning using AI has traditionally been avoided, in part because it requires vast amounts of data.
But there’s enormous potential, says Petch, and the CREATE team was able to study reinforcement learning by accessing huge amounts of data from warfarin studies through partnerships with Harvard and Duke universities in the United States, Uppsala University in Sweden and PHRI, a joint research institute of HHS and McMaster University, a global leader in large clinical trials.
Through this partnership, CREATE gathered the largest data sample ever assembled for the purpose of studying warfarin dosing. “We wanted to explore whether AI could do a better job of helping to get the dose just right on a continual basis,” says Petch.
And while the study focused on warfarin, its results could apply to other conditions, such as diabetes, where constant monitoring of medications is essential for accurate dosing.
“For these patients and their doctors, it’s a never-ending dance to stay at just the right range for optimal dosage management,” says Petch. “For example, foods high in vitamin K, like broccoli, can counteract the blood-thinning effects of warfarin, causing them to bounce in and out of the therapeutic range. The same is true for someone with diabetes, who may go in and out of the glycemic range if their blood sugar levels aren’t carefully managed.”
Partnering for innovation
Harvard, Duke, Uppsalla and PHRI had run randomized control trials for alternative medications to warfarin. These studies included control groups of patients that continued taking warfarin as their standard of care for comparison’s sake against new drugs being tested. Those control groups were what interested the CREATE team, because they provided data from hundreds of thousands of encounters with doctors responsible for monitoring the warfarin-taking study participants and adjusting their warfarin dosages as needed.
Using this data, the CREATE team was able to use reinforcement learning to train an AI model to recommend warfarin dose changes. It turned out that AI did very well – providing dosage recommendations at the same level as the very best, most highly skilled physicians involved in the studies.
“By simply observing this enormous cycle of data, AI showed that it could learn to dose warfarin as well as the very best, most experienced and highly skilled clinicians who specialize in this area,” says Petch.
Until the CREATE study, no one had rigorously shown that reinforcement learning with AI may have an important place in health care.
“Our study has opened that door,” says Petch. “As well as showing the potential to develop a whole new and better approach for dosing these kinds of medications, we’ve helped establish that using this paradigm of AI can work in health care with enough data. That’s what really interested us and the wider scientific community.”
