Load balancing is a computer networking method for distributing workloads across multiple
computing resources, such as computers, a computer cluster, network links, central processing
units or disk drives. Load balancing aims to optimize resource use, maximize throughput,
minimize response time, and avoid overload of any one of the resources. Using multiple
components with load balancing instead of a single component may increase reliability
through redundancy. Load balancing is usually provided by dedicated software or hardware,
such as a multilayer switch or a Domain Name System server process.
Internet-based services One of the most commonly used applications
of load balancing is to provide a single Internet service from multiple servers, sometimes known
as a server farm. Commonly load-balanced systems include popular web sites, large Internet
Relay Chat networks, high-bandwidth File Transfer Protocol sites, Network News Transfer Protocol
servers and Domain Name System servers. Lately, some load balancers have evolved to support
databases; these are called database load balancers.
For Internet services, the load balancer is usually a software program that is listening
on the port where external clients connect to access services. The load balancer forwards
requests to one of the "backend" servers, which usually replies to the load balancer.
This allows the load balancer to reply to the client without the client ever knowing
about the internal separation of functions. It also prevents clients from contacting back-end
servers directly, which may have security benefits by hiding the structure of the internal
network and preventing attacks on the kernel's network stack or unrelated services running
on other ports. Some load balancers provide a mechanism for
doing something special in the event that all backend servers are unavailable. This
might include forwarding to a backup load balancer, or displaying a message regarding
the outage. Load balancing gives the IT team a chance to achieve a significantly higher
fault tolerance. It can automatically provide the amount of capacity needed to respond to
any increase or decrease of application traffic. It is also important that the load balancer
itself does not become a single point of failure. Usually load balancers are implemented in
high-availability pairs which may also replicate session persistence data if required by the
specific application. Round-robin DNS
An alternate method of load balancing, which does not necessarily require a dedicated software
or hardware node, is called round robin DNS. In this technique, multiple IP addresses are
associated with a single domain name; clients are expected to choose which server to connect
to. Unlike the use of a dedicated load balancer, this technique exposes to clients the existence
of multiple backend servers. The technique has other advantages and disadvantages, depending
on the degree of control over the DNS server and the granularity of load balancing desired.
Another more effective technique for load-balancing using DNS is to delegate www.example.org as
a sub-domain whose zone is served by each of the same servers that are serving the web
site. This technique works particularly well where individual servers are spread geographically
on the Internet. For example,
one.example.org A 192.0.2.1 two.example.org A 203.0.113.2
www.example.org NS one.example.org www.example.org NS two.example.org
However, the zone file for www.example.org on each server is different such that each
server resolves its own IP Address as the A-record. On server one the zone file for
www.example.org reports:
@ in a 192.0.2.1
On server two the same zone file contains:
@ in a 203.0.113.2
This way, when a server is down, its DNS will not respond and the web service does not receive
any traffic. If the line to one server is congested, the unreliability of DNS ensures
less HTTP traffic reaches that server. Furthermore, the quickest DNS response to the resolver
is nearly always the one from the network's closest server, ensuring geo-sensitive load-balancing.
A short TTL on the A-record helps to ensure traffic is quickly diverted when a server
goes down. Consideration must be given the possibility that this technique may cause
individual clients to switch between individual servers in mid-session.
Scheduling algorithms Numerous scheduling algorithms are used by
load balancers to determine which back-end server to send a request to. Simple algorithms
include random choice or round robin. More sophisticated load balancers may take additional
factors into account, such as a server's reported load, least response times, up/down status,
number of active connections, geographic location, capabilities, or how much traffic it has recently
been assigned. Persistence
An important issue when operating a load-balanced service is how to handle information that
must be kept across the multiple requests in a user's session. If this information is
stored locally on one backend server, then subsequent requests going to different backend
servers would not be able to find it. This might be cached information that can be recomputed,
in which case load-balancing a request to a different backend server just introduces
a performance issue. Ideally the cluster of servers behind the
load balancer should be session-aware, so that if a client connects to any backend server
at any time the user experience is unaffected. This is usually achieved with a shared database
or an in-memory session database, for example Memcached.
One basic solution to the session data issue is to send all requests in a user session
consistently to the same backend server. This is known as persistence or stickiness. A significant
downside to this technique is its lack of automatic failover: if a backend server goes
down, its per-session information becomes inaccessible, and any sessions depending on
it are lost. The same problem is usually relevant to central database servers; even if web servers
are "stateless" and not "sticky", the central database is.
Assignment to a particular server might be based on a username, client IP address, or
be random. Because of changes of the client's perceived address resulting from DHCP, network
address translation, and web proxies this method may be unreliable. Random assignments
must be remembered by the load balancer, which creates a burden on storage. If the load balancer
is replaced or fails, this information may be lost, and assignments may need to be deleted
after a timeout period or during periods of high load to avoid exceeding the space available
for the assignment table. The random assignment method also requires that clients maintain
some state, which can be a problem, for example when a web browser has disabled storage of
cookies. Sophisticated load balancers use multiple persistence techniques to avoid some
of the shortcomings of any one method. Another solution is to keep the per-session
data in a database. Generally this is bad for performance because it increases the load
on the database: the database is best used to store information less transient than per-session
data. To prevent a database from becoming a single point of failure, and to improve
scalability, the database is often replicated across multiple machines, and load balancing
is used to spread the query load across those replicas. Microsoft's ASP.net State Server
technology is an example of a session database. All servers in a web farm store their session
data on State Server and any server in the farm can retrieve the data.
Efficient but simple approaches are a very common case where the client is a web browser,
per-session data can be stored in the browser itself. One way to achieve this is to use
a browser cookie, suitably time-stamped and encrypted. Another is URL rewriting. Storing
session data on the client is generally the preferred solution: then the load balancer
is free to pick any backend server to handle a request. However, this method of state-data
handling is poorly suited to some complex business logic scenarios, where session state
payload is big and recomputing it with every request on a server is not feasible. URL rewriting
has major security issues, because the end-user can easily alter the submitted URL and thus
change session streams. Encrypted client side cookies are arguably just as insecure and
unless all transmission is over HTTPS, they are very easy to copy or decrypt for man-in-the-middle
attacks. Load balancer features
Hardware and software load balancers may have a variety of special features. The fundamental
feature of a load balancer is to be able to distribute incoming requests over a number
of backend servers in the cluster according to a scheduling algorithm. Most of the following
features are vendor specific: Asymmetric load: A ratio can be manually assigned
to cause some backend servers to get a greater share of the workload than others. This is
sometimes used as a crude way to account for some servers having more capacity than others
and may not always work as desired. Priority activation: When the number of available
servers drops below a certain number, or load gets too high, standby servers can be brought
online. SSL Offload and Acceleration: Depending on
the workload, processing the encryption and authentication requirements of an SSL request
can become a major part of the demand on the Web Server's CPU; as the demand increases,
users will see slower response times, as the SSL overhead is distributed among Web servers.
To remove this demand on Web servers, a balancer can terminate SSL connections, passing HTTPS
requests as HTTP requests to the Web servers. If the balancer itself is not overloaded,
this does not noticeably degrade the performance perceived by end users. The downside of this
approach is that all of the SSL processing is concentrated on a single device which can
become a new bottleneck. Some load balancer appliances include specialized hardware to
process SSL. Instead of upgrading the load balancer, which is quite expensive dedicated
hardware, it may be cheaper to forgo SSL offload and add a few Web servers. Also, some server
vendors such as Oracle/Sun now incorporate cryptographic acceleration hardware into their
CPUs such as the T2000. F5 Networks incorporates a dedicated SSL acceleration hardware card
in their local traffic manager which is used for encrypting and decrypting SSL traffic.
One clear benefit to SSL offloading in the balancer is that it enables it to do balancing
or content switching based on data in the HTTPS request.
Distributed Denial of Service attack protection: load balancers can provide features such as
SYN cookies and delayed-binding to mitigate SYN flood attacks and generally offload work
from the servers to a more efficient platform. HTTP compression: reduces amount of data to
be transferred for HTTP objects by utilizing gzip compression available in all modern web
browsers. The larger the response and the further away the client is, the more this
feature can improve response times. The tradeoff is that this feature puts additional CPU demand
on the Load Balancer and could be done by Web servers instead.
TCP offload: different vendors use different terms for this, but the idea is that normally
each HTTP request from each client is a different TCP connection. This feature utilizes HTTP/1.1
to consolidate multiple HTTP requests from multiple clients into a single TCP socket
to the back-end servers. TCP buffering: the load balancer can buffer
responses from the server and spoon-feed the data out to slow clients, allowing the web
server to free a thread for other tasks faster than it would if it had to send the entire
request to the client directly. Direct Server Return: an option for asymmetrical
load distribution, where request and reply have different network paths.
Health checking: the balancer polls servers for application layer health and removes failed
servers from the pool. HTTP caching: the balancer stores static content
so that some requests can be handled without contacting the servers.
Content filtering: some balancers can arbitrarily modify traffic on the way through.
HTTP security: some balancers can hide HTTP error pages, remove server identification
headers from HTTP responses, and encrypt cookies so that end users cannot manipulate them.
Priority queuing: also known as rate shaping, the ability to give different priority to
different traffic. Content-aware switching: most load balancers
can send requests to different servers based on the URL being requested, assuming the request
is not encrypted or if it is encrypted that the HTTPS request is terminated at the load
balancer. Client authentication: authenticate users
against a variety of authentication sources before allowing them access to a website.
Programmatic traffic manipulation: at least one balancer allows the use of a scripting
language to allow custom balancing methods, arbitrary traffic manipulations, and more.
Firewall: direct connections to backend servers are prevented, for network security reasons
Firewall is a set of rules that decide whether the traffic may pass through an interface
or not. Intrusion prevention system: offer application
layer security in addition to network/transport layer offered by firewall security.
Use in telecommunications Load balancing can be useful in applications
with redundant communications links. For example, a company may have multiple Internet connections
ensuring network access if one of the connections fails. A failover arrangement would mean that
one link is designated for normal use, while the second link is used only if the primary
link fails. Using load balancing, both links can be in
use all the time. A device or program monitors the availability of all links and selects
the path for sending packets. The use of multiple links simultaneously increases the available
bandwidth. Shortest Path Bridging
The IEEE approved the IEEE 802.1aq standard May 2012, also known and documented in most
books as Shortest Path Bridging. SPB allows all links to be active through multiple equal
cost paths, provides faster convergence times to reduce down time, and simplifies the use
of load balancing in mesh network typologies by allowing traffic to load share across all
paths of a network. SPB is designed to virtually eliminate human error during configuration
and preserves the plug-and-play nature that established Ethernet as the de facto protocol
at Layer 2. Routing
Many telecommunications companies have multiple routes through their networks or to external
networks. They use sophisticated load balancing to shift traffic from one path to another
to avoid network congestion on any particular link, and sometimes to minimize the cost of
transit across external networks or improve network reliability.
Another way of using load balancing is in network monitoring activities. Load balancers
can be used to split huge data flows into several sub-flows and use several network
analyzers, each reading a part of the original data. This is very useful for monitoring fast
networks like 10GbE or STM64, where complex processing of the data may not be possible
at wire speed. Relationship to failovers
Load balancing is often used to implement failover—the continuation of a service after
the failure of one or more of its components. The components are monitored continually,
and when one becomes non-responsive, the load balancer is informed and no longer sends traffic
to it. When a component comes back on line, the load balancer begins to route traffic
to it again. For this to work, there must be at least one component in excess of the
service's capacity. This can be much less expensive and more flexible than failover
approaches where each single live component is paired with a single backup component that
takes over in the event of a failure. Some types of RAID systems can also utilize hot
spare for a similar effect. See also
References ^ "High Availability". linuxvirtualserver.org.
Retrieved 2013-11-20. ^ IPv4 Address Record
^ Shuang Yu. "IEEE APPROVES NEW IEEE 802.1aq™ SHORTEST PATH BRIDGING STANDARD". IEEE. Retrieved
2 June 2012. ^ Peter Ashwood-Smith. "Shortest Path Bridging
IEEE 802.1aq Overview". Huawei. Retrieved 11 May 2012.
^ Jim Duffy. "Largest Illinois healthcare system uproots Cisco to build $40M private
cloud". PC Advisor. Retrieved 11 May 2012. "Shortest Path Bridging will replace Spanning
Tree in the Ethernet fabric." ^ "IEEE Approves New IEEE 802.1aq Shortest
Path Bridging Standard". Tech Power Up. 7 May 2012. Retrieved 11 May 2012.
External links Dynamic Load Balancing and Scheduling