>> She might be an artist and we're just trying to give her the paintbrush.
You know she might be an author and we're just trying to give her the
ability to put words in sentences together.
>> Good morning ladies.
Give him his new pencil please.
>> I was telling Grace, oh, cool, the kids are being nice and calm this morning
didn't I? And then
two seconds later..raerrrr.
Aden was picking on Sam. Sam was screaming. Aden was hitting
Sam with a football.
See, Grace is, she's funny.
She's beautiful. She loves to play games.
She loves to be around people. She's very social. She was probably close to six
months old when we found out
We questioned a lot of her delay in her
process of, you know, sitting up, and even smiling.
Actually, cerebral palsy never even came to mind when
any of this was occurring, until we took her to the doctor.
>> For the past two academic years we've used the Microsoft Kinect
to build games and apps for kids with autism.
This academic year we decided to try things a little bit differently.
>> So we have a very special guest rockstar here today.
This is Grace Simon. >> Over the summer we thought about what if there is one
particular child that we could work with
and if she could actively participate in the class, come and visit the class.
If the students could meet this child, if they could ask her questions,
than that would just be hugely beneficial. We're trying to come up with
different modes of her to interface and be able to communicate.
>> Grace, when you're in class
and you have a question for your teacher, are you able to ask that question?
Yes okay now how do you ask the question?
>> Grace, with your chart with the letters,
is it difficult for you to point at letters that are that small?
Or is that a good size? That's a good size?
>> Would you like to send email?
>> Email's kinda getting old. Maybe texting?
>> That's so 1990's.
>> I'm sure all of us are really motivated to do a really good job on
this project. We all want to help her as much as we can.
>> I think, this class helps to open your eyes. We're all very involved with
modern technology and like the advancements in computing. But it kinda
opens your eyes to the fact that there are these subsets of
the population that are excluded from it.
>> This is the draw-free app. For example if she wants to
indicate that she wants to go for a walk she could just put
people representing her family members and also some scenery.
What makes this different from other paint applications
is that up a lot less dependent on controlled
hand movements that are required for moving a mouse or drawing free hand.
>> Grace what do you think of the button? Do you not like the button?
>> No
>> I don't like it either. >> Maybe a different button...
Like if it was easier to press, I think..
>> This project, it's socially
relevant because we're helping Grace, but
it's also a really interesting engineering problem.
Okay, so as an example, we're trying to do eye tracking.
Okay but we're trying to do eye tracking on Grace
while her head is is moving around a little bit, so
you know you can maintain eye contact with something but if you're maintaining
eye contact with you know a letter on a virtual keyboard
but your head is moving around, it's hard when the frame of reference changes.
>> We want to create a decision tree. A computer science tree that you can
navigate and every
single category will have subcategories that will reflect that above category.
>> Essentially,
the difficult engineering problem is determining intent.
Intent you have to dwell on something, you have to spend some time
doing it.
If you want to type the letter A you reach out and you
hit the letter A. Okay but if your hand is moving around and it's hard to hit
the letter A, do you
make the assumption that that person wants to hit the letter A?
So the challenge for us now is to
kinda provide a safe, non-stressed
environment for Grace to use these different software tools.
So we wanted to move some of the games into her house.
>> Grace can essentially use these,
some of these projects
in her daily life and see what it'd be more like to actually have them.
She'll actually be able to learn over time.
>> A group of three to five students would work on a project for 14 weeks,
and then they move on. I mean a lot of them are going to work
at full-time positions at leading companies in the United States.
We are actively trying to create
products that will work for Grace and for kids like Grace.
So one of the things we're working on, in fact,
is to try to create some kind of
nonprofit corporate structure so that these products that the students have
created can live on past the students' time here at the University of Michigan.