Seizure Forecasting Using AI - Daniel Goldenholz, Harvard Beth Israel Deaconess Medical Center, MA, USA
Could we accurately predict epileptic seizures using AI? How accurate do predictions need to be? How cool to use do devices need to be?! Could seizure forecasting reduce injuries, decrease mortality, and improve people’s quality of life? Hear all about it from neurologist: Daniel Goldenholz.
Reported by Torie Robinson | Edited and produced by Pete Allen
Podcast
-
00:00 Daniel Goldenholz
You know, that's really sobering to know that I can take a group of people and say, I'm going to give you something dangerous. Do you want it? And they said, yes. Because it's very dangerous to give a wrong alert about seizure forecasting. It's different than, you know “Is it gonna rain today or tomorrow? Maybe I take my umbrella, maybe I don't. With a seizure, maybe I'll take my medicine or maybe I won't. Maybe I'll drive or maybe I won't. Maybe I'll go swimming or maybe I won't.” And these are life-threatening decisions.00:26 Torie Robinson
Fellow homo sapiens! Welcome back to Epilepsy Sparks Insights.
This is part 2 of 2 with neurologist and data scientist Daniel Goldenholz! Last week we spoke about ethics and biases in the development of epilepsy technologies and this week we speak of the thoughts of people with an epilepsy when it comes to seizure forecasting devices - including things like accuracy and alerts!
A quick one - please don’t forget to like, comment and subscribe. Your comment and like will help spread awareness and understanding of the epilepsies around the world.
Now, onto our star of the week, Daniel Goldenholz!01:00 Daniel Goldenholz
I am an assistant professor at Harvard Beth Israel Deaconess Medical Center. Part of my job is to be an epileptologist, so I read EEG, take care of patients on the inpatient side and have a clinic. And, a larger part of my job is to do epilepsy research, and that is focused on data science and AI; trying to apply some of the latest technologies in those domains into the fields of epilepsy.01:22 Torie Robinson
There are heaps of organisations out there trying to do seizure forecasting. Largely, in my experience, from my limited knowledge, looking at tonic-clonic seizures or tonic seizures, seizures of movement, rather than just focal seizures. What are your thoughts about the seizure prediction devices, limitations and potentials?01:43 Daniel Goldenholz
So, it's a big can of worms. Let's say this, that you know as a physician who takes care of people who are you know dealing with seizures all the time and as someone who has a family member with this condition, I know full well, we're not gonna wait. We want to know right now “When is the next time we're at risk? And this is not something that we'd like to someday have, it’s something we want to know now. If it's not perfect, give it to us anyway.”. There's been a lot of really exciting research over the past few years suggesting that, yes, it is possible to do seizure forecasting. The most famous example of this was - already 10 years ago - was published, a study where somebody put (well, a group of people put) some wires into patients heads and attached it to some hardware and then some software. There was a warning light, and there was a “You're having a seizure now” light, and then there was an “Everything's okay” situation. So, people loved having those. There was only 15 patients in that study, they lasted for a couple of years, the company ran out of money. And there was a beautiful paper, and there was many, many papers based on those patients since then. But that experiment has never been repeated - yet. And that experiment showed that, number 1: it is possible to give people a bit of warning. And it is... it's not perfect, we know that it's not perfect. So, number 2: it's not perfect! Number 3: people didn't care! They loved it! They loved having that peace of mind, that they knew that there was some machine that was saying, like “You're gonna be okay now” or “This is a little bit of dangerous time”, or “Yeah, that's a seizure”. People really, really appreciated having those pieces of information. And not only that people appreciated, but it's possible and it was doable. Since that experiment was done, the information that came out of those patients' heads has been studied many, many times. And future studies said “Oh, you know what, we could have done better than what they did at the time using the same information we could have done more sophisticated approaches and so on. So, I think that is sort of the biggest piece of evidence that this could work. It's possible that we can give people some kind of a forecast. But then you say “Wait a second, you got to stick a wire in my head in order to give me a ‘maybe’ probability that maybe I'm going to be able to have a seizure or not? I don't know about that, doc. That sounds a little bit crazy!”. And that's true, it is pretty crazy because of course you're going to have the risk for bleeding and the risk for infection and the risk for stroke if we do anything inside your head. So, do we really want that? Could we do something else? Maybe, like something you wear. Well, there's some really exciting research that says, yeah, maybe. Maybe you could wear something, and that thing can give you some idea about your risk going up and down over time. And those risks, again, they're not perfect, but maybe people would like them and maybe people would appreciate them and maybe it'd be worth it. So, then, even further, our group of several years ago said “Maybe you don't even need to wear anything. Maybe you just need to have a diary.”. And we came up with this way of doing that, and we made some very exciting claims that maybe it's possible just with the diary alone, you can make a risk for tomorrow. And that was very exciting, and to be very clear, what we said is “Mathematically it's possible, if it actually helps people, we don't know yet.”. And since then, we've looked more closely, and I think that it might be mathematically possible, but the way we did it wasn't as smart as we could have been, and we've gotten smarter over time.
I don't know if it's still possible. I hope so, but I don't know anymore. There are some new ideas that we didn't have at that time that might help. So, with all of that background, what's the deal? Can we do it? Yes, or no? And the answer is maybe.05:26 Torie
Hahaha.05:26 Daniel Goldenholz
If we can do it, it certainly is not going to be perfect. And the question is, to me “Is that okay?”. Can I give you, let's say, I don't know, and it doesn't matter, let's pretend that you have active seizures. If you have active seizures, can I give you this tool that I know for sure will fail you sometimes? Are you okay with that? And we did a study where we asked that kind of question to a group of people with active epilepsy. And to my surprise, they said “Yeah, we'll take it! We don't care that it's not perfect. If it tells us too many alarms or not enough alarms, give us that tool.”. And, you know, that's really sobering to know - that I can take a group of people and say “I'm going to give you something dangerous. Do you want it?” And they said “Yes”. Because it's very dangerous to give a wrong alert about seizure forecasting. It's different than, you know “Is it gonna rain today or tomorrow? Maybe I take my umbrella, maybe I don't. With a seizure, maybe I'll take my medicine or maybe I won't. Maybe I'll drive or maybe I won't. Maybe I'll go swimming or maybe I won't.” And these are life-threatening decisions. So, I think that having a seizure forecasting tool that's not super-accurate raises - when we talk about ethics - it raises a real serious ethical question on the part of the investigator or the company that offers it: “Are you sure this is safe? And if you're not sure that it's safe, do you have a way to communicate to your patients that you know for sure they're gonna understand?”. Because if not, somebody’s gonna get hurt. And I worry, ever since I got started in this field, I got really worried about that, and I'm still worried about it. How can we provide these forecasts in a responsible way to people who...…sometimes don't make good decisions even without AI.07:11 Torie Robinson
I'm totally with you on that. I'm very up in the air: I've been involved in studies with other people with epilepsy and some people would say, well, if I'm gonna get an alert and that does not prove accurate, I don't have a seizure or I'm not actually going to have a seizure, you're gonna make me so stressed that I will have a seizure that I will have a seizure, even though ordinarily I might not have had that seizure. And yeah, so there are so many different sort of… type of.. angles [at which] you could look at it. I think accuracy is a big deal. It's a really, really big deal. That's what holds lots of, in my experience, lots of people with epilepsy back. But I really liked the point you made, actually, about people with epilepsy or their carers, actually understanding the, at least current limited accuracy. I think it's really important.08:05 Daniel Goldenholz
You know, it's funny, I did an informal survey of just random people that I spoke to, and I said “Listen, do you use a weather app?”. And this is something that a lot of people do, right?08:14 Torie Robinson
Yeah, I do.08:14 Daniel Goldenholz
They have an app on their device, and it tells them, is it gonna be snowing, is it gonna be raining, is it gonna be a hurricane, is it gonna be whatever. And I said “Why the heck did you use that one? Because there's many apps out there.”. And this is actually the same kind of question for seizure forecasting “Why is it okay to use this tool, which is inaccurate?”. And when it comes to weather apps, the answer I got from most people is the same. They said “Eh, I like this one.”. In other words “I put about 5 seconds of thought into the video, and I came to the conclusion, this one is okay.”. And I asked them questions like “Well, does it ever give you the wrong answer?”. “Yeah, sometimes it does.” So, they know it's inaccurate, and yet they're willing to stick with it. Why? It's not always inaccurate. And also, it doesn't matter that much. So, I think that the calculus is probably different when it comes to a seizure forecasting tool. And I make a big fuss with other investigators about the difference between something that is mathematically useful and something that's clinically useful. And I think that mathematically useful, I think that this has been proven that we can do that, which is amazing. We can do something that might give us a little bit of an edge over randomness. That's amazing, and I'm so excited about that. That means that we're on the right track. But if it's not better than something that you can do in your head, on your own without some fancy tool, then it's still useless.09:33 Torie Robinson
And, also, simple things that might seem not that important, but like alerts. So, another conversation I had with somebody; it was actually about the voice alert. I said “I would not use that because his voice, for whatever reason; creeps me out…09:48 Daniel Goldenholz
Hahaha.09:48 Torie Robinson
…Really creeps me out.”. And they said “Well, what type of alert would you like?” I said, well, do you know what? I think I'd like some - yeah, this is dark humour - but, for instance “Who Wants To Live Forever” by Freddie Mercury? Like that would be funny to me. Um, or, and other people are like “Oh my God, you're sick!” “Well, yeah, I am, but to me that works!”. Do you know what I mean?10:07 Daniel Goldenholz
Sure.10:08 Torie Robinson
And so, if it was just this creepy voice, I wouldn't use it because it would stress me out. But those are the types of things I think, you know, if you're just working in academia alone and you haven't spoken to people, you wouldn't know that, right?10:20 Daniel Goldenholz
And that goes back to the weather app, right? I mean, there are some weather apps that I don't like because I don't like the pictures that it shows me. I don't like that it provides the information in a dumbed-down way or in a too-complicated way. I want it just right. And you wanna hear an alert that sounds pleasing and not frightening. And I think that all that stuff is important. The psychology of how should we be warned is not a small thing. And most of myself and my colleagues that are trying to do this. We're not focused on that, to be honest with you. We're focused on going from a little bit better than random to useful, clinically. And I think that jump is such a big one that we haven't even had a second to think about, well, what kind of voice should we use and what colours are nice and that sort of thing. And I say that, and at the same time, I mentioned Sharon Chiang, she's actually done some studies and others have done as well, on what would be the right way to show to patients. And they've done focus groups and they've shown different examples of things. So, I think that people have thought about this a little bit, and I think that we're gonna keep on, you know, iterating through this, and everyone's gonna have their own “What do I like?”. But getting back to the math thing for just a second, it turns out that if I propose to you whatever your seizure rate was before will still be your seizure rate tomorrow. That mathematical trick is extremely powerful. So “If you have a seizure every day, here's my forecast for you. Tomorrow you'll have a seizure.”.11:47 Torie Robinson
Ha!11:47 Daniel Goldenholz
And it sounds idiotic, but mathematically beating that prediction is much harder than just a random coin flip. And so, I want patients and investigators to know that this one thing: predicting that tomorrow is going to be like the last few months, that's hard to beat mathematically. And if we can beat that, then we're on the right track to something that's clinically useful.12:09 Torie Robinson
Thank you to Daniel for seeing seizure detection devices from the perspectives of clinicians, researchers, and the person with an epilepsy - and their family as well actually. It’s a challenge but is absolutely crucial to combine insights from each party when it comes to seizure prediction device development.
If you haven’t already, don’t forget to like, comment, and subscribe, and see you next time! -
00:00 Trailer & intro
01:00 Meet Daniel
01:22 Accuracy in seizure forecasting devices
05:26 Must devices be 100% accurate?
08:05 Weather apps in comparison
09:33 Simple things are crucial to users
11:27 Mathematical challenges of seizure prediction
12:09 Conclusion & thank you to Daniel
-
Daniel Goldenholz is an assistant professor of neurology neurologist, epilepsy specialist, and data scientist/epilepsy researcher at Harvard Beth Israel Deaconess Medical Center in MA, USA.
His research is focused on data science applied to the field of epilepsy for diagnostics (multimodal imaging and biosensor techniques), therapeutics (clinical trial studies) and prognostics (seizure forecasting). Daniel’s long term goal is to “find ways to help end the suffering of patients with epilepsy.”
-
Goldenholz Epilepsy + Data Science Lab: goldenholz-epilepsy-data-science-lab
Harvard Catalyst: Person/27784
LinkedIn: daniel-goldenholz
Harvard Catalyst: Person/27784
ResearchGate: Daniel-Goldenholz
VJ Neurology: daniel-goldenholz