Hi.
I'm Josh Urban Davis from Dartmouth Discovery Lab.
My slides will allegedly be here eventually.
Ah!
There we are.
OK.
So yes.
Hi, I'm Josh Urban Davis.
And today, I will be presenting our work on CircuitStyle,
a system for peripherally reinforcing
best practices in hardware computing.
Let's talk about hardware computing.
Novice engineers often learn how to prototype circuits
by replicating tutorial circuits on breadboards.
However, teaching and learning circuit prototyping
is difficult for a variety of reasons,
including that novice engineer circuits tend to be big,
stinking, wiry messes that are very difficult to understand
and even more difficult to debug.
This creates frustration for novice users
and can often be discouraging from them
pursuing an understanding the discipline.
To solve for this, we look to software engineering.
And we're inspired by the practice of coding styles.
Examples of coding styles include
type indentation-- such as tabs versus spaces,
naming convention-- such as camel case versus underscore,
and bracket etiquette.
Coding style can be thought of as good penmanship for coding.
You will notice in the two slides here--
the slide on the left has implemented a particular kind
of coding style that makes it easier to read
than the slide on the right.
You might not agree with that because one thing that we
discovered was that much like fashion style or graphic design
style, coding style is subject to subjectivity.
And often, different instructors and different practices
employ different style practices in their coding.
But what we found, regardless, is
that implementing coding style practices in an organization
can greatly increase the readability of code,
as well as make the code less prone to errors.
So our inquiry was quite simple.
Many instructors agree that similar best practices exist
for circuits and are important to learn but rarely are
they taught in formal settings due to time and attention
constraints placed upon the instructor.
To account for this, we conducted a design and query
and prototyped a web interface called CircuitStyle--
a peripheral behavior reinforcement system
for in-person hardware computing tutorials.
We devised a semi-structured interview
and a series of formative studies
by recruiting five instructors who
taught electronic prototyping at various institutions including
formal primary education classrooms,
makerspace workshops, and higher education.
We compiled a list of best practices
behaviors which you can see a sample of here,
and there are more in the paper.
And we asked these instructors to rank on a Likert scale
from 1 to 7 the importance of these behaviors.
To give you an intuition of the kind of best practices
we are referring to, I will show you a few now.
One in particular that was very important to instructors
was ensure a component's polarity is
correct before insertion.
For any of those in the audience who have ever blown up an LED,
you are very familiar with this particular pitfall.
Another best practice that we discovered that was popular
was avoid changing components on a breadboard whenever
the board is powered.
Because this can be quite dangerous
and you can give yourself a bit of a shock.
This work builds upon existing research
in the design of classroom management and learning tools
such as Maestro which was presented previously
here at UIST--
circuit prototyping tools such as trigger-action-circuits
and insights from teaching coding styles.
Our research goals are to better understand these best practices
for hardware and to examine how we
can design an interface to gently encourage students
to make better implementation choices when building
a circuit with an instructor, without further burdening
that instructor to reinforce these practices.
I'll now provide you with a brief walkthrough of our web
interface system which is comprised of three components.
First is a tutorial authoring tool for instructors.
This allows instructors to author their tutorial step
by step--
uploading photos for each particular step.
This is to encourage instructors to complete
the tutorial themselves while they are authoring it,
so they can take photos and identify problem spots
that students might encounter.
Once the tutorial has been authored,
we then allow the instructor to either use our default
repository of circuit style behaviors
or to author their own circuit style behaviors.
In either case, either editing or writing their own style,
there are several customization aspects
to writing a circuit style behavior.
This includes uploading photos of both good and poor
implementations of that particular behavior,
as well as a description of the behavior--
indicating a severity level for the behavior
from low, medium, or high, and choosing a method
to reinforce that behavior such as either
using a quiz which the instructor will write
or to use peer review--
which we will address later in the student
section of this presentation.
Once the instructor is satisfied with the repository of styles
that they have authored, they may then assign these styles
to particular steps where keeping that style in mind
would be useful.
We also provide a very crude recommendation system
for assigning these styles to particular steps.
We'll now examine the second portion of our web interface,
which is an in-situ interface for instructors
for in-person workshop tutorials.
This is the main interface that the instructor
sees during the in-person workshop tutorials.
At the bottom of the screen, you'll
see a series of pie charts.
Each one is associated with a particular circuit style.
And the red and green reflects the class's overall performance
at that particular behavior.
At the center of the screen, you will notice the circuit diagram
for the current step that the instructor is demonstrating.
And on the right, you will notice
a list of all of the steps in this particular tutorial that
are color-coded according to the class's overall progress
through the tutorial.
Red, for instance, indicates that very few students are
currently completing that step.
Green indicates that the majority
of the students in the class are completing that step.
And gray indicates that no students are currently
completing that step.
By clicking the style button in the top right-hand corner
of the screen, the instructor is given a more granular view
of the classroom's overall performance.
This grid is associated with the number of cells, each one
with a particular student in the class.
For each student, the instructor can
see this student's most recently uploaded photo for peer review,
as well as a pie chart reflecting
how well that student is performing on the circuit style
behaviors for the overall tutorial.
The instructor can also provide positive reinforcement
and direct feedback.
The third and final element of our overall user interface
is a student view for these in in-person workshop tutorials.
The first step after logging in is the students
are asked to sort out the steps of the tutorial
themselves-- the tutorial that the author writes.
Each of the steps is in randomized
and the students are asked to sort them.
This is done in order to encourage students
to keep in mind that there is a proper and safe procedure
for implementing circuits.
I return you to our earlier intuitive example
of making sure all of the elements are in the breadboard
before powering it.
This step was designed in order to encourage
these sort of behaviors.
After completing the sorting step,
the student is then brought to the main interface
for the tutorial.
On the right, you will see the current step
that the student is completing--
working on, with implementation instructions, as well as
a photograph-- detailing how to complete that step.
And on the left, you will see the style guide
which is color-coded according to the severity
of that particular circuit style behavior
and contains both good and bad examples of how to implement
that particular behavior.
It is collapsible, as well as searchable.
And the default style is always indicated by the instructor
during the authoring step because it
may be pertinent to this particular step.
After completing each section of the tutorial,
the students are asked to take a photo
of their current breadboard implementation.
This is then sent to other students for peer review.
After the student has taken the photo
and sent it to the other students,
they are then presented with a photo of another student's
work, as well as a good example of particular circuit style
behaviors and asked to comment on whether or not
those behaviors are evident in this particular breadboard.
Once the student has completed this,
it is sent back to their peer to give them
feedback on their performance.
And they also receive feedback from their peers
on their performance of these style behaviors.
This is done in order to reinforce these behaviors
without creating an additional cognitive load or labor
burden on the instructor.
To evaluate the effectiveness and usability of the system,
we developed a case study with undergraduates.
We recruited 11 students for a 90-minute instructor
led workshop where the students were
asked to create a button activated motor circuits.
We collected a variety of information and data
on the performance of these students.
The instructor was asked to create the tutorial beforehand.
And during the tutorial, we collected audio and video data
of the student's progress through the tutorial,
as well as conducted semi-structured interviews
with both the participants and the instructor,
and also completed entrance and exit questionnaires including
a Likert scale exit questionnaire
to evaluate the overall performance and interaction
with this interface.
Some of our key takeaways is that glanceable awareness
of student's progress was very important to our instructor.
We didn't want the instructor to spend their entire time staring
at a computer screen.
We wanted them to engage with the students
as much as possible.
And using this color-coded system
that was immediately recognizable,
allowed the instructor to see as quickly as possible
where the students were and what they might be having trouble
with.
And this encouraged them to address these issues directly.
We also wanted to supplement and not
replace an in-person instructor's capabilities.
We think the interaction between an instructor and a student
is very, very important.
And we found much to our surprise
that the instructor commented that they
found more often than not, the interface
helped them to go immediately to the students who
needed help and also were able to better facilitate
that interaction with them.
Finally, we wanted our system to have an ambient nature.
It needed to sit in the background
and not obstruct the student from learning
nor require the instructor to conduct additional labor,
in order to teach this particular kind of material.
We also collected feedback from students.
I should mention now that 100% of our students
were able to complete the tutorial.
We allotted 90 minutes for the workshop
and the students on average took 41 minutes and 16 seconds,
with a 9-minute and 14-second standard deviation.
So in about half the time what we allotted,
they were able to complete the circuits.
And we found that only three out of the 11 students who
use the style guide and the workshop
required additional assistance outside of the instructor
or the interface.
In summary, what we can take away from our case
study is that best practices for hardware computing
are important to learn.
And in addition, these best practice behaviors
can peripherally be reinforced with carefully designed
software that doesn't create an additional burden
on the instructor-- by allowing students
to better reinforce the material through peer review
with each other.
This work is very preliminary and provides many opportunities
for future work in this area of inquiry.
One thing is scaling peer review would be very useful.
We found that students often--
when they were performing peer review--
would turn to their neighbor and ask them
what they thought of the particular circuit
that they were working on.
And this is something we really think should be encouraged,
not discouraged by our particular system.
So adjustments to our system could
help to account and encourage this sort of in-person peer
interaction.
And our system in our current implementation
might be better suited for larger classroom sizes.
In addition, we wanted to account for varying skill
levels with our students.
Some students expressed insecurity in the feedback
they were providing their peers since they were novices--
they had never done this before.
And so they were kind of worried that the feedback they
were giving was not very good.
So future implementations and adjustments
in this particular kind of work could calibrate the system
based on the student's knowledge and previous experience--
allowing more experienced students to mentor
lesser experienced students.
And finally, AR based approaches could reduce the code switching
required for a student to go from the computer screen
to the hardware prototype and could instead
present the entire interface in an augmented reality
glass in front of them.
I'd like to quickly take this opportunity
to thank the UIST organizing committee,
as well as our session chair, Elena Glassman.
And I would also like to thank my collaborators, Jun Gong
and Xing-Dong Yang from Dartmouth Discovery
Lab, as well as Parmit Chilana from Simon Fraser.
I'd also like to thank our workshop instructor,
the interns, the participants, and everyone
who made this particular project possible.
I think I somehow ended well under time.
That never happen during the rehearsal.
Great.
So I now would be happy to take your questions.
You may also ask me by email or on Twitter.
Thank you so much.
[APPLAUSE]
So I'd like to ask whether you see this more as kind
of a Checklist Manifesto that people are going to go forward,
making better designs with?
Or if you see it more as something
to be drilled into students' heads
such that as they go off into the world it's internalized,
and they continue to follow the best practices--
and these are the training wheels that you eventually
expect to takeoff their bike?
That's an interesting question.
I love the phrase Checklist Manifesto.
But since style behaviors in coding and programming
in particular vary from organization to organization,
we have a feeling that hardware style behaviors, as well,
will vary from organization to organization.
And so the goal overall is to just make sure that--
kind of gently encouraged students
to pay attention to these sort of things.
And so when they do go out to these organizations,
they can implement these circuits
in a cleaner way that's more readable by their peers
and easier for them to figure out what's wrong
when something does break.
That's right.
I see.
Going to assume I just explained everything very well.
You did.
All right.
Hey!
Sorry, I'll ask a really obvious question.
How far would you see this scales?
How much did the instructor actively
work with the students?
Could this scale to a hardware MOOC or something?
Yeah, that's a really interesting question.
Yeah.
The instructor worked-- we really
did want to encourage the instructor and students
like in-person interaction.
Because we found that to be-- we think
that's extremely valuable and especially,
learning these kind of fundamental skills.
We think it could definitely scale to larger classrooms.
But the instructor really was--
we were worried that the instructor was
going to spend the entire time staring at their computer
screen and not helping students.
And we found that they actually didn't really
do that very much at all.
He would come by on occasion and glance at the screen
and then go straight off to another student
to help them work.
And this was really one of the goals of our design decisions--
to make sure that the instructor was
spending as much time interacting
with the students as possible.
But seeing how well it would scale to 200 person intro class
would definitely be an interesting thing
to examine in future work.
Thank you for the question.
If we don't have any other--
I think we have a third question.
We must.
Yes!
Thanks.
Great talk.
Awesome stuff you got there.
Thank you.
So I'm David, from University of Calgary.
One question that I have is in terms of scale,
not in terms of people, but in terms of the types of hardware
prototypes that you make.
Because I think-- one big challenge
that I noticed especially with Arduino type of stuff
is that it's really [INAUDIBLE] one small thing to work,
but once you build a larger object then it becomes a mess.
So I was wondering, how do this could
go into more complex objects?
And whether you've seen that or looked into that.
Yeah, totally.
That's a great question.
And we actually addressed this in our limitation section
in the paper.
And I was going to address in the limitation section here
and took it out last night.
But one of the features that we had
was breaking it down into particular sections
and subsections to be completed.
So we can take a large circuit and break it down
into smaller and smaller and smaller components
and then peer review those smaller components-- as
opposed to looking at and trying to reinforce
these practices in a huge, huge system that does something
big and complicated.
And our circuit was very, very simple.
All it was was to push a button and motor turns on.
And in the future, it would be really, really useful
to actually understand if this is scalable to these larger,
more complex circuits without creating one of the messes
that I showed at the beginning of the talk.
Thanks.
Thank you.
Wonderful.
Thank you again.
Thank you.
So I have two announcements.
The first is that please do-- as you're watching these
presentations both in this session and beyond--
indicate which ones you think