But the problem doesn't stop there. Metadata is
overloaded, but my video has gotten popular. And
so all of these guys want to download
the content from me, which means that everybody
is coming to me, and I have an itsy bitsy server in my basement. And that
is going to get inundated with all of
these download requests from all of these different clients
that want to get this video from me. So, there is a metadata overload and there
is a content overload that is coming about
because of the fact that something became really hot
and everybody wants to get that content. And
this is what is called the origin server
overload. In general, there are two solutions to
the origin server overload problem. The first solution is
to have a web proxy. And you might
be familiar with this already because every organization tends
to have a web proxy so that they can limit the amount of requests that have to
go out of an organization. And that way if users of an organization
are trying to access popular content from the Internet, the local web proxy
that's available in the organization can directly serve
that content to the requesters. But it turns out that
the web proxy solution is not good enough for what
is called a Slashdot effect. And that is, if there
is a breaking news and everyone wants to get the
content, they will have to get the fresh content. A
proxy's not good enough for that. Let's say that there's
a Super Bowl going on and everybody wants to watch
the Super Bowl, or at least get the latest updates
from that. In that case, the web proxy's
not going to be good, because the content
in the web proxy is cached content. You
want the live content. And the live content is
only available at the origin server, not at
the proxies, and therefore the origin server will get
overloaded when we have such dynamic content. Content
that is resulting from either breaking news, or live
programming, and so on. And this is
where the content distribution networks come into play.
The idea is that the content is
automatically mirrored from the origin server at selected
geographical locations. And those locations are constantly
getting updated from the origin server. So that
going to any of the mirrored content is the same as going to the origin server.
And then what happens is that, in
the content distribution network, depending on the geographical
area from which a particular user request
originates, the user request is dynamically re-routed to
the geo-local mirror of the content so
that the origin server need not be overloaded.
You may be familiar with companies like Akamai
which have come about for the purposes of
providing content distribution network for popular
content providers, like CNN or broadcast television
channels like CBS, NBC, and so on. And what these organizations, that is
the content providers, do is get into an agreement with a content distribution
network provider like Akamai, so that they don't have to worry about origin
server overload because the content distribution network
provider like Akamai have the solution for
taking the content of the origin and
automatically mirroring it in geo-local sites so
that the origin server never gets overloaded. So, this is good if you are a big
organization like CNN, or CBS. You can afford to pay the content
distribution network provider to do this automatic mirroring, but
how can we democratize content distribution. How am I,
in my basement, generating a piece of video and
I want to share it with the world, how
am I going to make that available to everybody?
If it gets hot, I don't want to get
overloaded. I want the content distribution network to work
without having to pay big bucks to a content
distribution network provider like Akamai. And this
is where the Coral System comes in. This
is a paper that I've assigned you
to read which has a technique for democratizing
content distribution. And the details of the
Coral System, and what we're going to look
at next. So the Coral System addresses two
issues. The first issue is the fact that
if I generate some content, and I want to store the key value pair
associated with the content in a DHT, I have to have a scalable way of
doing it without saturating any particular node
which can serve as a metadata server. So
we want to avoid the tree-saturation effect and the second thing we want to do
is also avoid the origin server overload.
So both of those things are being addressed
in providing a democratic solution for content
distribution for the average Joe that may want
to generate some content, and want to share it with the rest of the world.
We will look at the details of the Coral System in the rest of this lecture.