so our next presenters are ready we are
having dr. David Malan and Kareem Zidane
from Harvard University dr. David J
Malan is a Gordon Mackay professor of
the practice of computer science at
Harvard University
he teaches computer science 50 otherwise
known as cs50 which is Harvard
University's largest course one of young
university's largest courses and edx is
largest massive online or massive open
online course he also teaches at harvest
Harvard Business School Harvard Law
School
Harvard Extension school and Harvard
summer school Kareem zidane is a
software developer system administrator
and teaching fellow for cs50 at Harvard
University he is a self-taught
self-taught programmer from Egypt who
discovered computer science including
cs50 itself online he is the chief
architect of the cs50 IDE the fun fact
for the both of them when asked what
food would they never get tired of
eating dr. David responded seafood and
Kareem responded Italian food please
join me in welcoming dr. David Malan and
Kareem is a Dane
[Applause]
I see it there but not here okay thank
you all so much I'll admit this is kind
of intimidating coming just minutes
after a talk on whether we're alone in
the universe our talk is I dare say a
little more down to earth we're here to
talk about integrated development
environments and my name is David Malan
I teach an introductory course at
Harvard called cs50 computer science 50
which is our introduction to computer
science for majors and non-majors and
I'm here with a colleague Kareem's of n
who was actually a student of ours
online years ago but has since joined us
in Cambridge Massachusetts to work on
the very set of problems that we're here
to talk a little bit about so I thought
I'd tea up what it is we do and what
problems it is that we aspire to solve
and Kareem will walk us through some of
these solutions to that ultimately so at
the start of the semester we have some
800 students in this undergraduate class
it happens to be Harvard's largest class
predominantly non majors but computer
science is now the second largest major
at Harvard and so it's a pretty large
demographic of students we have each
term the course we similarly make freely
available to anyone online not only as a
massive open online course via EDX at
this URL including all of the materials
all of the software all of the sample
solutions and the like but also all of
the courseware is available as open
courseware in the spirit of mi t--'s
tradition such that anyone on the
internet can download and access any of
these materials at all beyond though
being just available to students online
we focused really on the community
aspect and building up local support
structure so to speak around the world
each red dot above represents a high
school that happens to have adapted or
adopted this core cs50 in some form and
each of the blue dots represents a
university where some faculty or student
group or nonprofit in the area has
similarly done the same to provide a
localized in-person support structure
for students following along and
teaching themselves computer science the
scale then that we operate on is fairly
large not only on campus but off on
campus we might have a thousand students
among the undergraduates students
Harvard's Extension school and also down
the road at Yale University where we
have a cohort of students as well and
then online over the past several years
have we had some 1 million registrants
in the course not all of whom engaged at
any given moment
but quite a few do in fact on any given
day might we have several thousand
students using the courses
infrastructure cloud-based
infrastructure that will soon address
and over the course of a month we might
have some 30,000 active participants
banging on the servers and the virtual
machines and now the containers that we
use to support those students work to
give you a sense of where it all began
though it all began much more modestly
much more locally in Cambridge
Massachusetts so back in the day late
eighties early nineties we just had a
non clamp on-campus cluster of servers
that students each had home directories
on usernames and efest Mountain home
directories in which they could do their
work our class is predominantly software
focused we tend to use tools like GCC
and gdb and the like if familiar to
focus on C and then later in the
semester do we transition to other
languages like Python so it's a Feist or
Leone and to this day to really just
give students a terminal window and so
back in the day they all tell NetID -
until the world figured out SSH at which
point the SSH to centralized servers
where they would do all of their work
and submit all of their work and this
was implemented on campus with a bunch
of UNIX servers in front of which was a
load balancer of some sort and then
students were round-robin distributed
among the load there the fact though
that the university ran this meant that
frankly the software was always several
years out-of-date we as an individual
course did not have an administrative
access or pseudo access to any of the
accounts so it was hard frankly to
support students optimally if we just
didn't have the software and the access
that we wanted to to be able to answer
questions help them died diagnose bugs
and the like and so cloud computing
caught our eye pretty early on when AWS
was just gaining steam for us back in
2008 we hopped on board ec2 the Elastic
Compute cloud and at the time there were
terribly few services on Amazon's menu
there was no elastic load balancing
there was no auto scaling or the like
anything you wanted to build you had to
build yourself by wiring together some
of the more primitive services and so we
did exactly that pretty much reaaahhr
tech ting that same topology this time
though in the cloud gaining for
ourselves pseudo access and the ability
to scale it up and down depending on the
day or the night of the week when a
homework assignment was due however
there was not without costs in fact I
think we appreciate it a little too long
in the process that the
thing Harvard was providing was system
administrators who used to run all of
this for us so all of a sudden now it
was us working not only 9:00 to 5:00
during business hours but also 5:00 to
9:00 when the students were actually on
these systems doing their work
particularly late into the night so not
without some hidden human costs early on
also at the same time because of the
OpenCourseWare community we had no
ability to provide students online
really with their own Harvard University
counts just by nature of not being able
to scale and for reasons of privacy and
security so they weren't able to tell
neder ssh to the same clusters on campus
or off and so we eventually transitioned
to a client-side virtual machine and
appliance that students could download
and so for several years we gave them
VirtualBox or VMware fusion or
workstation would like so they
downloaded in environment on to their
own Macs or PC that they then did that
all that same work it gave us
additionally a nice graphical interface
which opened up new problem set
capabilities since we were no longer
restricted to just a command line
environment but it too was not without
its challenges frankly a VirtualBox back
in the day wasn't terribly reliable it
was not uncommon for students at the end
of the work and in the day in middle of
a pset close their laptop and boom
virtual hard disk bricked because it
wasn't unmounted correctly and so those
kinds of headaches and Windows and Mac
OS particulars just got in the way of
maintaining a common infrastructure for
students in a way that just worked and
so we transitioned ultimately to a
cloud-based ID in 2015 and that's where
we are now and where the story picks up
where we went back into the cloud and
this time though focused more on
software is a service using an
open-source tool called cloud 9 formally
implemented by a team at a startup now
part of Amazon Web Services and began to
add some pedagogically motivated
JavaScript based plugins to this
environment and ultimately to build out
the backend that provided students with
the same illusion that they had access
to their own account or really their own
service in the cloud for those
unfamiliar this is a screenshot of cloud
9 or specifically cs50 s incarnation
thereof with some pedagogical
simplifications and UI modifications it
provides students with a familiar tab
based code editor like you would find on
any modern system provides them most
importantly with the terminal window so
they have full fledged access to the
underlying system a file browser at top
left but then also some more teacher
friendly and programmer friendly
features like the ability to chat in
real time with someone with whom they
appear programming or a teacher who
might want to help
remotely it provides them with an
interactive graphical debugger that
provides them with the ability to step
through their code line by line the
ability to roll back in time a la
version control but this time
graphically so students can see where
they left off just minutes ago and
ultimately here's the screenshot of that
same debugger that allows them to stick
to step through stack frames and the
like but Amazon is cloud 9 - out of the
box connects only by default to ec2
servers and it's designed to do this so
that you start up your own ec2 server
you run this web-based UI and from there
you get to do anything you want and it
by way of this cloud-based IDE so it was
up to us though to decide how best to
provide students with this same UI we
didn't want them signing up for AWS with
their own credit cards and the like
administering their own servers because
we would be regressing back to that
point where we didn't have that same
access ourselves so we tried a number of
versions of solutions to this to build
out a cloud-based back-end for Amazon's
front end including ec2 for compute and
s3 for storage then s3 alone or EBS for
storage
then s3 alone for persistent storage and
so finally did we happen on kubernetes
and docker containers more generally
which was an extension of what we were
already starting to do on campus for our
own internal development and that's the
story where Corrine will now pick up as
to the problems we encountered along the
way and the solutions we ultimately
forged using kubernetes and containers
for cs50
thank you David so among the reasons why
we dealt with cloud 9 as the base
software to see if the CCI de is that
one it has a nice plug-in model and it
allows us to extend the software in a
way that was quite handy for students
who are first coming into computer
science or programming it might be
overwhelming for them to see the actual
interface of cloud 9 or
other IDs so let me explain briefly how
client 9 can kind of works cloud 9 so
has the UI that we saw a moment moments
ago and then there's the compute
resource that this UI connects to where
the actual programs are running where
the actual files are store and this
connection happens to be an ssh
connection of course by default in case
of cloud 9 this connection this this
compute resource happens to be an ec2
instance by the pole so our first
approach was to basically build our own
very simple very basic orchestration
system that uses ec2 for compute and
uses EPS for storage so we would have a
pool of ec2 instances and a bunch of
users in some availability zone in AWS
where each user gets one EBS volume and
then we would basically reuse these easy
two instances or life terminate them
bring some up between between different
users of course if we zoom out a little
bit a little bit more we have more than
one availability zone each region we
have multiple regions that we want to
operate in ok so let's walk through some
of the high-level key steps in this
implementation details we would generate
a federated user
it's basically generating credentials to
grant these users access to certain
resources that we want them to use in
our own Elba's account because this is
all running in our own account of course
we would get unavailable ec2 innocence
from the pool and the reason being is
that it makes the process faster we have
innocences ready to be used that we can
assign to user direct directly we would
create the EBS volume and then attach it
to the innocence we would format the EBS
volume so create a file system and so on
and then we would mount a volume in the
instance we would run some commands to
start the docker container to all
Rize the public SSH key for the cloud 9
environment to connect to this container
we would get the public hostname for the
ec2 instance and then finally we will
connect the cloud 9 environment to the
innocents or to the container in this
instance and then redirect the user to
the environment so this is just so you
have a mental picture of how this is all
implemented we have the cloud 9
environment the front end connecting by
SSH I want a container running an easy -
innocence and then we use EBS for
persisting students file for storage all
right so what are some of the challenges
that we ran into when using this
approach well first of all we had to
maintain a pool of ec2 instances across
different availability zones across
different regions which which wasn't
quite easy to do like we had to maintain
the state on some metadata about this
pool of institutions we had to get down
to a level of running commands on the
ec2 instances directly to start the
containers to authorize the public key
using another Amazon Web service called
SSM which wasn't quite abstract enough
we had to allocate an entire ec2
innocence for each student for each
cloud 9 environment even if they're not
using the entire ec2 instances in terms
of resources CPU and memory and such and
then at the end of the user session we
had to terminate the ec2 instances which
was a bit time-consuming because we had
to also wait for the EBS volume to be
detached and another challenge with that
is that when the user comes in next time
for a new session they would get a
different hostname which was quite
annoying maybe for students who were
working on the ROI project getting a
different host name every time and it
was also quite challenging for us
because the cloud 9 environment now
needs a different parameters for this is
H connection and so final final thing is
that we had to remove the ec2 instances
temporarily from the pool to update the
docker images that are back in the IDE
ok what are some challenges that we had
with EBS for storage so again we use a
ps4 persisting students files in the IDE
we had to also provision and allocate an
entire EBS volume for each student even
if the student is not really using the
entire EBS volume which wasn't quite
cost-effective and we have to assign
every user a single availability zone
which was quite limiting because that
means that
even if in the pool somewhere else in
some other availability zone there was
an innocence available
we couldn't assign this innocence to the
user because it had to be in the same
availability zone as the EBS volume and
so we in an attempt to resolve the
storage challenge we moved to s3 for
storage instead and while this actually
held sort of eliminate this limitation
of having to be in the same availability
zone it introduced a few challenges of
its own so here's a mental picture for
the same architecture basically with s3
used for storage instead of EBS and some
of the challenges that this will
introduce so we basically create a
folder or prefix on an s3 bucket for
each student and then we would grant
them read and write permission to this
prefix one of the challenges is that we
had to set up credentials to be able to
download this data and upload this data
periodically and we had to refresh this
these credentials across sessions it
would be nice if we didn't have to do
that of course we download the users
data initially which was quite
time-consuming it made the IDE slower to
start we had to upload the data
periodically which wasn't quite robust
so anything could go wrong and in the
user we lose some data potentially and
then we could not find an easy way to
limit the size of this prefix on s3 by
design s3 is probably meant to be sort
of flexible in terms of how much you can
store and so we could not really limit
stored in these prefixes so at this
point we started considering other
services for for our implementation and
all we really wanted is a service that
we could ask to give us some container
that ideally has a fixed hostname that
we can use that has persistent storage
that's reachable from the internet so we
consider services like AWS for gate I
know because I manage our orchestration
system on AWS but unfortunately two
downsides of AWS for gate is that it
doesn't support image caching and that
made it slower to start IDs as well when
we tested that it took anywhere from two
to three minutes to start each container
which was very slow and it doesn't
support persistent storage so that was a
problem with AWS for GAE another service
that we considered is AWS ECS which is
a little bit more manageable s managed
version of a debates for gate where we
actually own and manage a bunch of
container hosts but we were using ezs
for some other application at the time
and we found the limitation that wooden
wasn't gonna play nicely in this
architecture the limitation that we
found is that the more concurrent
containers or tasks that we had running
in ECS the slower it takes to start new
ECS tasks or containers and that of
course was going to affect the idea in
this case and then finally we considered
kubernetes so kubernetes is another open
source orchestration system and then
here are some steps where we create an
IDE on with with the kubernetes approach
so we created for a user like before to
grant users permission to use some
resources we create a namespace per user
and then we create a persisting of
persistent volume claim this is
basically claim for storage we ask for a
certain amount of storage to be
available for this user we authorize the
public usage key like before we create a
single container pod or a container we
mount the persistent volume on it we
mount the ssh key to be able to
authorize processes connection and then
we connect the cloud 9 environment and
final redirect the users so what
challenges did we have with kubernetes
let's talk more about what solutions we
that actually cupen is help us solve so
previously where we had to maintain this
pool of easy to instances across their
availability zones and regions
kubernetes actually manages the nodes
for us so if a node comes down if an ec2
instance comes down it will bring it
back up automatically which is quite
nice well previously we had to go down
to the level of running commands on each
ec2 instances we don't really have to do
that anymore because you use the
kubernetes api to ask for a container
and it just we don't worry about how
this container is created at all how the
volumes are mounted and so on and so
forth we used to allocate an entire ec2
instances for instance for each user and
now we're actually able to run multiple
containers on one Innocents and have
them share resources ultimately and at
the end of the session we used to
terminate each ec2 illnesses and wait
or EBS volume and get a different host
name next time and everything and right
now we actually have some deterministic
hostname per user no matter what
container they use thanks to some
reverse proxy that's implemented as an
application running in the same
kubernetes cluster and the cluster is
DNS as well and finally we used to
remove the ec2 innocence to update the
docker image remove the CC to innocence
from the pool to update the docker image
well now we can do that very easily
using something called daemon kubernetes
which is essentially a container that
runs on every instance every work and
node that we have in the cluster and
effectively pulls the image down so
these are some challenges that we had
with storage ABS ns3
and we resolve these challenges using a
tool called port works and again storage
persistent storage on kubernetes is not
an easy thing to do as well and so port
works really helped us sort of solve
some of these problems and the way port
works works is that it basically creates
a set of EBS volumes for us we don't
really care about how to how it creates
them we give it just the credentials to
create them we don't care which region
we don't care which availability zone
and then it gives us basically a storage
pool so a virtual storage pool that we
can just use to allocate virtual volumes
from and then finally we use we reuse
the capacity in this storage volume by
basically taking snapshots to s3 of
these virtual volumes deleting them and
then later when the user comes in we
basically create the virtual volume
again from this snapshot another
advantage of port works is that storage
is thinly provisioned and that means
that if we ask for one gigabyte for one
user for example it's not necessarily it
doesn't necessarily mean that we're
gonna lose a one gigabyte of the storage
capacity until the user is actually
using one gigabyte so that was quite an
optimal thing to have and so here is how
this is implemented here's a diagram of
how this is implemented on the
kubernetes cluster we have the cloud 9
environment like before connects to a
container or in situ innocence via SSH
except that we have a proxy
right now in the middle that helps
resolve this DNS issue and
to have a deterministic hostname for
each user no matter what container
they're using and then this is of course
all part of a cluster and then we use
port works for storage all right so I
guess one main takeaway here from from
this talk is that we are kind of using
kubernetes in a way that's not that's
not usually that it's not usually used
for but it's lending itself really
nicely to our implementations lending
itself really nicely to our solution
here's some here's some graph for the
number of users that we see since July I
think and we can see anywhere from 4,000
to eight and a half maybe thousand users
every day which is quite scale plays
really nicely on scale and here's some
future work as well so one of the one of
the challenges that we have with this
implementation even with the previous
implementations is fraud detection and
you know maybe more more monitoring so
it turns out that some users some bad
people out there in the world use the
IDE for things that it's not meant to be
used for for example to bet coalmine or
to do some remote server somewhere and
so we want to get better really lifting
this we are using AWS gateway and other
Amazon Web service that helps us monitor
some of the traffic that's going into
and out of the cluster and we have some
other mechanisms in place as well to
monitor some file names and processes
names and so on
we want to use multiple clusters across
different regions and finally we want to
have multiple IDs a per user which is
quite easily I think to implement in
this model all right so that's it for
the cs50 IDE I encourage you all to go
ahead and try it out at IDE . cs50 the
ILO let me invite david back on stage to
take some questions if we have time and
thank you so much