source: Wikimedia

Healthcare is hard. This week we talk about recent challenges at IBM’s Watson Health and take material from one of the very first Cloudys.

Healthcare is challenging

Just to get into the field is hard. Research scientists can spend 5+ years getting their PhDs. Physicians can spend 10+ years specializing in a field. Finally, if you are a sadist, you can enter into a program to get an MD/PhD, which means you may never leave school…

Even if you make it out of school, it doesn’t get easier. If you dream of creating new therapies that will cure all sorts of diseases, you are fooling yourself.

Last year was a good year for drug approvals in the United States, and that number was 46. This is actually up close to 100% from the 10 year average, which was 22.9 between 2000 and 2010. However, that doesn’t mean we are getting better at drug discovery. If you actually look at a chart going back to 1995, there really hasn’t been any change in new drug approvals per year and we are far off the record of 1996. What was so special about 1996? Hmm, maybe the Olympics being in the USA?

US FDA drug approvals by year

What about AI coming and saving the day?

After the success of Watson on Jeopardy, IBM focused its AI efforts on healthcare and was off to a quick start. Watson health created many high-profile partnerships and set out a mission to help physicians treat patients using AI.

Of course, there were roadblocks. MD Anderson ended a praetorship with IBM due to costs and just recently, IBM announced layoffs within the Watson Health division. While some of the layoffs were standard practice as IBM acquired some companies to build the division, it doesn’t bode well for IBM after negative press last year about its progress.

But if everything healthcare related is hard, applying AI to healthcare should as be hard?

Let’s say this year you are filling out a World Cup bracket and you are tired of listening to talking heads on television who always get it wrong. You collect some data on previous tournaments and build a model to predict the outcome. Your model performs well, and you crush your office pool. After the tournament, you are looking for some excitement, so you shift your focus to major league soccer and use the same model. Or maybe you have a daughter who plays soccer and you want to impress some other parents by predicting the outcome of the game, so again you use that model. Soccer is soccer, right? People run around, kick a ball into a net, what could be different?

Apply the same logic to healthcare

Now, let’s say you just had one goal, to predict the type of leukemia a patient has (Leukemia was an original focus which IBM tried to apply AI to). If you just start at the basics and Google “number of types of leukemia”, you get four. If you pick a type of leukemia, chronic lymphocytic leukemia, and start to read into it, things become tricky. First, the course of disease depends on what types of mutations the cancer has, which according to one study, there are 55 known mutations. But during that study, researchers found two previously unknown mutations. Another paper from 2016 claims “There are an estimated 20-50 additional mutated genes with frequencies of 1%-5% in chronic lymphocytic leukemia; more work is needed to identify these and to study their significance”

So, just to reiterate, we only think we know half of the mutations that exist for one cancer.

Oh, and I forget to mention that as the disease progresses, the types of mutations you have can change. We have no idea why.

But, ok, how different can the same disease with one mutation be? Let’s just try and predict the original four.

Different therapies work for different mutations and some therapies don’t work at all. In fact, some therapies actually make things worse If you have just one particular mutation. If you really want to be effective, we have to predict what type of cancer someone has, then what sort of treatment they will receive…

…and even if we know everything, depending on the therapy, only 15% of patients will actually be helped by it. We’ll probably need more AI for that too…


Yes, hmm indeed

Watson Health did have some success after switching their focus to predicting a type of lung cancer. The Watson system was still able to provide 90% accuracy and physicians anecdotally mentioned they were impressed with the system and were excited for the future.

Of course, that future will take a long time and require a lot of research, money and patience. Articles like this, will continue to be released questioning the direction and progress of Watson Health, until we see a breakthrough

And that will be hard

Facebook Comments