In architecting the data store and the computational
resources that back the data store at a server,
the system administrator has a choice of either
replicating or partitioning the data. If you replicating the
data what that means is. That every data
server has the full corpus of data that is
needed to serve a particular request. Think of it
as say, gmail. If it is gmail, then if
a gmail request gets sent to anyone of
these servers. They can handle the request that comes
in because they have the full corpus of data
to deal with a incoming request. We mentioned that
failures are inevitable in giant scale services because
of the number of computation elements that live within
a data center. If failures occur, what happens when
we have replicated data? What they means is that
the harvest is going to be unchanged because if
some data repository is failed. Because it is
replicated, we can redirect a client's request to
another live server that has full access to
the later repository and therefore the harvest that
you're going to get with replication in 100%, it
is unaffected. On the other hand because the
server capacity has come down due to these failures,
the heal is going to come down. In other
words, with replication. The service is unavailable for
some users for some amount of time. But all the users that are able to get the
service, get complete harvest in term of the
fidelity for the query that is returned and answered
by the server. The other alternative of course
for architecting the internal data depository of the server
is to partition the data. So if you
partition the data, and let's say that There are
end partitions of the full compass of data. And
let's say that some of the service fail, then
some portion of the corpus of data becomes unavailable.
If this happens, what it means is that the
harvest is going to be done because some portion of
the data corpus. Is unavailable, and therefore the harvest
is going to come down, but it is possible to keep Q unchanged, so long as the
computation capacity is unchanged. Then we can serve
the same number of user community. It's just that
the fidelity of the result may not be as good as when All of the data corpuses
available. For example, if this is a server
that is serving search results and if some portion
of the data repository is unavailable. In
that case, each query that comes in will
get service, but it is going to get. A
subset of the query results for the search
that they are doing. So the important message
that I want to convey through these pictures
is that dq is independent of whether we
are replicating or we are partitioning the data.
Given a certain server capacity, The D Qs
are constant, because we are assuming that this
is cheap. This space is cheap and therefore
processing incoming requests are not disc bound, but
it is network bound. The only exception is
if the queries Result in significant in right
traffic to the disk. Now this is rare
in giant scale services. But if that happens,
then replication may require more DQ than partitioning. But as a first order of
approximation, so long as we assume that
giant scale services are serving client requests which
are network bound, The DQ is independent of the strategy that is used by the
system administrator for managing the data whether
it is replication or partitioning. Now what this
also means is that this is giving
an object lesson to the system administrator. In
saying that beyond a certain point, you
probably want to replicate and partition. Why? Because
failures are bound to happen, and a
failures are bound to happen if you partition
the data, then a portion of the
data corpus becomes unavailable. On the other hand,
if beyond a certain point Even if
you have partitioned data, if you replicate it,
then you're making sure that, even if one replica is down, some of the replica,
of the same partition is available for
serving a particular client request. And replication, beyond
a certain point is important because Users would
normally prefer complete data or a complete harvest.
Take for instance, you're accessing your email through
the Internet. You would rather put up with
a service being unavailable for some amount of
time rather than seeing that you only have access
to a portion of your main box. And
that's the reason why replication beyond a certain
point is better for dealing with giant scale
services. On the other hand, for searches, it may
be okay to have a harvest which
is not complete. So in structuring different services
as a giant scale service, one has to make a decision. Of whether partitioning is
the right approach or at what point we may want to partition and then replicate
as well. As a concrete example, Inktomi which
is a CDN server uses partial replication It
uses full replication for email because of what I just said, and that is
a user would expect complete harvest for a service like email.
But on the other hand, if it is a web cache that is
being implemented as giant-scale service, it is okay if it is not replicated.