Sign up and stay in-the-know about The Crowd & The Cloud and the world of citizen science.
My name is Pietro Michelucci. I direct the Human Computation Institute.
How did you get started on EyesOnALZ (formerly WeCureALZ)?
I'm trying to find big problems to solve, and a colleague put me in touch with Chris Schaffer. We had a conversation, and he said, "We're trying to cure Alzheimer's disease. This is the way we're doing it." It sounded really, really exciting and promising, so I said, "Why are we talking?" He said, "Well, it takes us, for each experiment, about one year just to analyze the data. That makes the cost of each experiment very, very high, so we need some help." It turned out there was a way to help.
It was all very serendipitous the way it worked out. I asked Chris, "Can you tell me, can you show me physically? How does this analysis work? What are the scientists in your lab doing to analyze the data that's taking a year?" He started showing me some pictures, and he said, "Well, what we're trying to do is we're looking at these blood vessels, and we're trying to figure out whether blood is flowing through them or not, but we have many, many vessels. Sometimes it's easy to tell, sometimes not so easy, so we have to have more than one person to do it. We have to check the answers against each other to make sure we have exactly the right model. If we don't have an accurate model, then we can't answer our research questions correctly." As soon as he showed me that first picture, in the back of my mind I'm thinking, "I'm already familiar with this. I have done this."
Interstellar dust particles embedded in aerogel, identified by Stardust@Home volunteers.
Stardust@Home was is a completely different project. We (NASA) were trying to find bits of dust from interstellar space that's embedded in aerogel. Those little dust trails look a little like blood vessels, and they created this virtual microscope. You use that to go up and down through these layers of aerogel. That's exactly the kind of work that Chris was doing in his lab. They're going up and down through layers of tissue in the brain to see whether blood was flowing through these vessels. As Chris continued to explain this to me, I got increasingly excited.
I called Andrew Westphal, who is the Principal Investigator on Stardust@Home. I described the whole problem to him. I said, "We're trying to cure Alzheimer's disease. Chris is doing this groundbreaking work. It's a little off the beaten path, but it's got the support of the scientific community." Andrew came back and we exchanged some images. He said, "You'd be surprised. I actually get inquiries like this from time to time. I almost always say no, but in this case, you're exactly right. This is dead-on a good fit for what we have. I'd be pleased to help you personally make this work, because I care about this."
How does Eyewire fit in?
I immediately went back to Chris, and I said, "I've got some great news. I spoke with Andrew and the Stardust@Home piece looks like it's going to work really well." Chris said, "Wait. There's another piece to this puzzle. Once we understand which blood vessels are flowing or stalled, we have to place that in the context. To answer our research questions, we need to know how these blood vessels are connected to each other. What the impact of a stall in this one blood vessel is on the rest of the blood vessels, downstream effects,” as Chris puts it.
I said, "How do you go about that? How do you build this model?" He showed me the pictures they take and how they have to manually look at those pictures and then build this 3D model of thousands of blood vessels to answer these questions. As soon as he showed me, it was like deja vu all over again. I looked at it and I said, "I can't believe this, but we're doing this, too. This problem has been solved."
At that point, I immediately thought about the Eyewire project where they're trying to map the human connectome, the neural wiring of the brain. I thought, "Networks of blood vessels, networks of neurons they're both networks. Maybe we can use this."
Then we went and we jointly approached Sebastian Seung and Amy Robinson, who are the progenitors of that system. It was a very productive conversation. There was a lot of enthusiasm on their end. They said, "I think it's worthwhile pursuing this. Let's see if we can make this work as well.” Then, we had the two pieces that we needed.
The most exciting part is when Chris and I sat down and we did the napkin math. We were trying to figure out: how much is this going to speed up the research? When we realized that a year's worth of research could be compressed into two weeks, and the impact that was going to have on the progression of the work, obviously we were thrilled. We were very excited.
How will the crowdsourcing analysis of these images actually work? What's the model to involve citizen science in this activity?
In a general sense, someone who wanted to make an impact in disease research, until very recently, would donate money to a research foundation. That way, the experts who do the work would be sustained so they could march that research forward. That's still a viable model today, but there are some problems where it doesn't matter how much money you throw at the problem, there's no way to advance that unless you put human minds to the problem.
Chris and his team tried different machine-based approaches, artificial intelligence, to do this kind of analysis. They got pretty close. They got an 85 percent model. That's not good enough. They need a 100 percent model in order to answer the research questions accurately. They had to do the manual work in the lab.
Today, there's an opportunity for people to contribute in a new way. That is to actually put their own cognitive abilities to this kind of problem. Frankly, as scientists, we're completely dependent on the general population to help us solve these problems. It used to be in villages, if you had a problem that you couldn't solve, the village would get together. Today, the Internet has made that village a planetary village. Up until recently, we didn't have a way to combine those planetary resources.
What we're doing is we're taking these tasks that the technicians do in the lab, and we're putting it online. We're gamifying it. We're making it quite simple and easy so anyone can go on and use this natural perceptual process that's evolved over millions of years to attack this problem. A person can go online and spend a couple of minutes making a decision about whether they see blood flowing through a vessel or not, and those few minutes multiplied by 10,000, 100,000, a million people, gets the job done very quickly.
Why do we trust that citizen science data could be sufficiently quality-controlled to guide academic and medical research that's literally life and death?
There are citizen science projects out there today that are already doing a good job of this. What they do is they've figured out ways of combining answers from a number of different people in order to produce on expert-like answer. There's a project called MalariaSpot, which is a fantastic project. It's an app. I use it myself. You go on and you count parasites on a blood smear. That helps diagnose malaria for a real person in Sub Saharan Africa.
Here's the catch. You wouldn't trust just me alone to do that. I've got two minutes of experience doing that. What they found out by doing a validation study is that if you combined on average, 23 people, and the responses from those 23 casual gamers, then you end up with one expert-like response, which is just as accurate as a trained pathologist. For this project, we're going to have to do our own validation study and figure out what that number is for us. How many answers do we need to combine from different people about the same blood vessel to have an answer that's just as accurate as the answer that comes from that trained laboratory scientist?
Imaging used to identify "stalls" in blood vessels, used for Alzheimer's research.
Where are you in the life of this project? What's happening in the next year?
**Pietro was interviewed in early 2016. Since then, as you can see in program 1, they’ve made considerable progress and have released the “StallCatchers” module, based on Stardust@Home.
This is a very exciting time in the project. It's also one of the most challenging periods in the project because everything is getting started all at the same time. There are lots of moving pieces. For example, we're working simultaneously on the Eyewire piece and the Stardust piece. We're at different stages in each of those. Each has different challenges associated with it. We're very excited because we already have a working prototype in the Stardust side of things. You can actually use an interface just like Stardust@Home to annotate a blood vessel.
This is so powerful that when Chris and his group saw the demonstration of this prototype, they said, "This would already be useful to us in the lab. We could use this today to already begin to speed up our process, even without the crowdsourcing piece."
At this point, we have goals, but there are a lot of unknowns. We're addressing various technical challenges along the way. We have to get the Alzheimer's data into a form that works in the Eyewire system. That's quite a complex process, but the same time, in a shorter amount of time we've managed to progress quite well through that. Now, we're probably looking at midyear (2016--and they made that target! C&C) for at least a beta release of one of the two parts of the project, either the Eyewire part or the Stardust part.
What would success for EyesOnALZ look like?
Project success means that we successfully analyze the data at least 30 times faster than they could've done it in the lab; that we can do that amount of work that used to take one year that the crowd can do in two weeks. To me, success is that we've arrived at a treatment target. Ultimately, everything we do is in service of that goal. This is a disease that doesn't just affect people. It affects families. It affects everyone. No one's immune from this disease.
One aspect of this that keeps me going is this is an exciting job. I love the people I work with. They're all game changers in their own fields. Like with any job, there are parts of it that are tedious. Sometimes you get out of bed in the morning and you say, "Gee, I got to go do this." That quickly goes away because as soon as I think about the fact that at this very moment, at any moment in time, there's a very large number of people around the world who are suffering from this disease, that's a quick wake-up call and a very strong motivator to get to work that day.
Does it matter in terms of this project that you may be not necessarily pursuing the hundred percent certified pathway to knowing what cures Alzheimer's?
In any research, we always begin with a question. That question leads to other questions. I'm very confident that Chris (Schaffer) and Nozomi (Nishimura) are on a very promising track with their research. In fact, I'd bet everything on it, myself, but it's research, and we don't know. If we discover something new about the disease from this, we're going to pivot, and we're going to pursue that goal. Fortunately, what this project does is enables the research to go in many different directions very quickly.
Here's the thing. One of the advantages of being able to run these experiments in two weeks instead of a year is that we can ask many more questions in the same amount of time. It also allows us to go down a path. Research often takes a circuitous path. It's like a dog. You think you're starting to sniff something interesting, and then all of a sudden you take a sharp turn. This happens in research all the time. What this allows us to do is be much more agile in that way, and regardless of what path this research takes us down, we can get to the answers much faster.
Life is full of moments, the things that matter in life. When you get to the end of your life and you look back, the things you remember are those special moments that you have with the people you care about. Alzheimer's disease starts to take those away from us, one at a time. It's not fair. I want to see that end.