hi everyone I'm Dante Gagne and I'm a Senior Product Manager here at Microsoft,
and we're here today to talk about inclusive development, and how you can empower developers
and their teams to achieve more. You'll meet Anna White, who's a Software Engineer at Microsoft,
and she'll tell you about the importance of save gestures and assistive tech.
Hi! I'm Anna White. I work as a Software Engineer right here at Microsoft.
Though I'm four years into my professional career, I've been coding games and websites since I was 12
years old. Computers are my passion. They're how I make, art stay in touch with my family, and create
really impactful products while earning my living. But, using a computer with my hands is painful. I
was born without a right hand due to a congenital birth defect called Amniotic Band Syndrome. But,
I never really thought about how computers weren't built for my body until I had to. Almost 10 years
exactly after I wrote my first line of code, I was diagnosed with severe repetitive stress injury,
early onset arthritis, and carpal tunnel in my left hand. Using a computer with my hand is
painful; just not the hand you'd probably guess. See, one in four computer users will develop a
chronic overuse injury. Having one hand, just means I ran into it twice as fast. Suddenly, every
click sent pins and needles shooting up my arm. From the minute I woke up, to the moment I fell
asleep (if I could fall asleep), the pain would not stop. I tried switching to using my shoulders
and my nub more to type, and my wrist got a little bit better. But six months later, I developed
chronic tendonitis in both of my shoulders. By that point, my arms were so bad I couldn't wash
my own hair or hold a cup. And every doctor I saw agreed that the root of the problem was the thing
I loved the most. My computer. Career computer use requires a lot of repetitive gestures, I
learned. More than any one part of the human body is designed to do long term. A paraplegic who uses
voice control will develop vocal strain. We've all probably met someone in our field whose hands
or lower back hurts at the end of the work day. Or, we are them. Or, we will be in 60 years. When
every click hurts, every click you can save, means all the more. And I tried all the hardware on the
market, and some of it helped a little. But, no matter what mouse you use, you still have to click
it. If even two-handed programmers develop chronic overuse injuries, what options are there for me.
But, programming is my dream job, my career, and I want it to be possible long term and today. Still,
I had to find a way to make what so many of us spend 40 hours a week putting our bodies through,
easier. I needed a way to reduce those painful gestures. I'm going to talk about two tools that I
honestly use literally every day, that help me do just that. The first you might already be a little
familiar with. And that's Intellicode for Visual Studio. Listen. My arms forced me to try a lot of
IDEs. I can say honestly that, Visual Studio is the best tool I have tried in getting code from
my brain to the screen with a little typing and clicking and dictating as possible. If you can't
use a keyboard without extreme pain, imagine how nice it is when I'm coding and Visual Studio pops
down its little suggestion list of what method to call on this object. But, Intellicode in 2023
is going even further than that. As I go through this demo, I want you to get a little bit weird. I
want you to put your arms at your side and really think about how long it would take you to do this
if you couldn't use them tomorrow, With that in mind, let's dive into a day of using my computer.
Test cases. Hard to design, but not hard to type out, usually. How often does this
happen. I add two new parameters to the validate method and now every test that references it
is broken. In big repos, this can be thousands of lines of code. Great. When my arms are bad,
I have to fix these mistakes with voice command: Go to End of Line. Press left key. Press left key.
Insert comma. Multiply that by 10. ... and you get the idea. So how does Intellicode make this
easier for me? Firstly, it's smart enough to read my code context and suggest up to an entire line;
even pulling in the error code variable I set above. Intellicode is trained on thousands
of public repos, so it's often able to suggest something that's actually useful to me. For the
parameter that didn't auto-populate, when I comma, Intellicode auto-suggests the error message string
first. This ordering is super important to me, because when it's right, all I have to do is press
tab. Intellicode has been doing a lot of work training its model to get that first suggestion
right as often as possible. We see Intellicode order the parameters usefully again as I add
the new ones to the call to validate. But now, I have to repeat this, 10 more times. UGH! But
look at what happens now. I do that edit a second time. And, on the third time, Intellicode is going
to do something amazing. It automatically suggests that same edit on the next line. Then, if I press
the preview all shortcut, (are you seeing this?), it applies the edit to every spot in my file that
matches that pattern. And, lets me take all those edits by just pressing tab. For me, this is huge.
It took fewer gestures to edit 10 lines, than it would take to just get the cursor onto the next
line without Intellicode. And, while this is great for me because typing is physically difficult,
if you have a focus-based disability like ADHD, a tool like this, could help you stay on track.
Another spot us developers get bogged down in, is copy and pasting code from other sources. Here,
I want to add in some M cell exception test code I copy over from a doc. I grab the code,
paste it in, and prep for a lot of: Press right key. Press down key. Paste. While I try to
rename all the parameters--is what I used to do. Watch this. I paste the code in, and Intellicode
triggers adaptive copy and paste. Are you seeing this? Press tab, and boom! Done. Whether copying
from a different spot in my own code, or pulling from an external source, this is such a time
saver. No more fiddling with find and replace. It's mind-blowing. It's also not mind-blowing,
because of course my IDE should do this for me. I can't believe it wasn't able to do this before.
And, these are just a few of the awesome AI features coming to Visual Studio in 2023.
It's hard for me to articulate just how impactful this is. I know a lot of people
look at these things and they think it's just saving you busy work, but not hard work. For me,
it is hard work. I see these features and I see all the time saved. All the gestures reduced. I
see a task my colleagues can do in minutes that used to take me 30 minutes. And now I can do it,
just as fast as them. I see myself being able to keep up and prioritize my time on the work that
matters. If I had something like this 10 years ago when I started coding, I probably could have
avoided a lot of my repetitive strain injuries. When I think of all the developers using Visual
Studio, and how many billions of gestures are being saved by this, it makes me so happy because
I know what it's like to be in pain every day and I want every techie to have as long a career as
they want with as healthy a body as possible, no matter what that body might look like. Artificial
intelligence like Intellicode for Visual Studio, is an essential item in our toolkit to help us
extend the lives of our hands, wrists, shoulders. Every one of us only has so many keys our bodies
can type before it can't. So, why waste it on the busy work. But, as you all probably know,
coding is a small fraction of what we use a computer for day to day. That's why Microsoft's
upcoming work in workflow automation across the whole computer experience is so exciting to me I
promise it's not just buzzwords I use a similar tool to what I'll be showing that I had to code
myself when my arms stopped working every single day it took enough load off of my arms that I
didn't have to go on disability leave even when I couldn't use a keyboard and that is AI powered
macros for everyone these are simple low or no code shortcuts that anyone can set up to automate
those tasks that are difficult or just repetitive think of it like a super cop customizable personal
assistant that you can trigger with your keyboard your Braille display your voice your mouse you
get the picture you tell it how you want to trigger it and what you want it to do when I
say open GitHub it switches to Microsoft Edge and loads github.com a person with a disability can
add their own macros unique to what they need to do and how they're able to do it which is
super useful if the how you use your computer is a little non-standard what you're going to see here
is an early beta I'll show you how I use it as a person with a Mobility disability we programmed
this assistant with some default commands that a lot of people might find useful like taking a
screenshot with your voice but me and Roberto both added in some specific commands for our workflow
and that is where the magic really starts to come through so let's dive back into my computer and
I'll show you how I use these AI powered macros to take the strain off my arms across my workflows
let's hop back into another code base here I have a syntax error but I can't remember the keyboard
shortcut to pull up the error window luckily I've mapped a macro to it show error window
hmm who on my team wrote this get blame maps to another keyboard shortcut and this one tells me
that my co-worker Rob should have more context the next macro is one of my favorites message
Rob why isn't this compiling the Handler switches to teams and pulls up Rob and as my conversation
in Nest text because this AI macro will run across your whole system it isn't tied to just
one application so you could build one of these to work with Discord or slack too voice isn't the
only way to trigger an action no company builds a keyboard that works ergonomically for my hands I
only have a left hand and Enter key is on the right so I set caps lock to trigger enter and
ta-da sending messages is so much easier but say my left hand's on the mouse and I want to
paste this code in my nub can't press two keys like Ctrl V at the same time here comes one of
our cooler triggers long press I hold the right control key and after a delay it pastes with one
key I also use long press as a trigger on all of my letters and number row to send the capitalized
or shifted version of a symbol which rocks since coding is so symbol heavy so it really gives my
left hand a much needed break I just remembered Rob mostly speaks German he'll have to translate
my message on his side or he used to translate this to German sends my selection to Ai and boom
ready to go saving Rob and Me time your macro can send data to a number of AI Services one I'll call
out now is well say Rob sends me a code example as this screenshot and I don't want to have to retype
it so I use a keyboard macro Control Alt s or I say take a screenshot with OCR which uses AI to
convert this image into text now I can paste it in with trustee long control and we are ready to go
look I am a software developer with limited arm Mobility our paraplegic graphic designer or a
blind digital music artist controls the computer very differently to do a very different set of
things we can't build one tool that makes all those workflows effortless but we can give
them a toolkit that they can tweak to make it work for them and that's how you create a tool
that can scale across the many many different shapes and types and kinds of people that use
their computers while making your own AI personal assistant might seem like Overkill to a lot of
people I feel its benefits every day I needed something like this to keep my passions alive
and I believe that a lot of other people with disabilities who can't find products built for
them and what they need to do feel the same our mission has always been to empower developers to
achieve more and the work our amazing presenters has shared with you today is just the beginning
Thanks again everyone, and take care