Hi!
Let's get started with the Using Cloudera Manager and Hue to Access a Big Data Service
Cluster video.
In this video, you will learn how to use Cloudera Manager (CM) and Hue to access a Big Data
Service (BDS) cluster.
To ensure security, when BDS deploys a cluster, the nodes in the cluster are not accessible
on the public internet.
By default, the IP addresses for the cluster nodes are private.
In this video, you'll learn how to access CM and Hue directly in a Web browser.
A Utility node generally contains utilities used for accessing the cluster.
For example, Cloudera Manager and Hue are installed on the first Utility node in a BDS
cluster such as Utility node 0 in our example.
You can make the nodes in the cluster available using one of the following methods:
You can map the private IP addresses of selected nodes in the cluster to public IP addresses
to make them publicly available on the internet.
We will use this method in this video which assumes that making the IP address public
is an acceptable security risk.
You can setup an SSH tunnel using a Bastion host.
Only the Bastion host is exposed to the public internet.
A Bastion host provides access to the cluster's private network from the public internet.
You can also use VPN Connect which provides a site-to-site Internet Protocol Security
(IPSec) VPN between your on-premises network and your virtual cloud network (VCN).
IPSec VPN is a popular set of protocols used to ensure secure and private communications
over IP networks.
Finally, you can use OCI FastConnect to access services in OCI without going over the public
internet.
With FastConnect, the traffic goes over your private physical connection.
Using a Bastion Host, VPN Connect, and OCI FastConnect provide more private and secure
options than making the IP address public.
The table on the screen shows some of the useful resources that you can use to establish
connections to the nodes in the cluster.
All services are only available on certain ports.
You need to define security rules on the subnet to allow access to those ports.
To make a service available, you must open the port for that service.
Port 22 for Secure Shell or ssh is opened by default in the default security list that
comes with each VCN.
All other ports are closed by default.
Security lists act as virtual firewalls for your Compute instances and other kinds of
resources with ingress and egress rules that specify the types of traffic allowed in and
out of the cluster.
Ingress security rules expose HTTP and HTTPS routes from outside the cluster to services
within the cluster.
To use a specific service on a node in the cluster, an administrator must explicitly
open the port associated with that service by adding an ingress security rule that allows
traffic into the cluster.
For additional information about security lists and rules, use the URLs at the bottom
of the screen.
The first requirement to enable CM and Hue to access the Utility node in our BDS cluster
is to open port 7183 for CM, and port 8888 for Hue.
We will add two ingress rules for CM and Hue to the default security list in our cluster's
VCN.
The second requirement to enable CM and Hue to access our BDS cluster is to map the Utility
node's private IP address to a reserved public IP address that you specify and provide your
users with this IP address.
For detailed information, use the BDS documentation URL shown on the screen.
As discussed earlier, there are other more private and secure methods to make the nodes
in the cluster available.
Login to the Oracle Cloud Console.
On the Sign In page, select your tenancy, enter your username and password, and then
click Sign In.
The Oracle Cloud Console Home page is displayed.
Click the Navigation menu in the upper left-hand corner of the Console Home page.
Under Networking, select Virtual Cloud Networks.
The Virtual Cloud Networks page is displayed.
Click your cluster's Virtual Cloud Network.
Next, we will update the default security list in our VCN.
Only public subnets can have a public IP address assigned to them.
Remember, when you created the cluster, you needed to specify that your cluster belongs
to a public subnet.
In the Subnets section on the page, click the public subnet for the selected network.
Under Resources in the panel on the left, the Security Lists link is selected by default.
Click your Default Security List link for your selected network in the Name column.
The Default Security List page is displayed.
Scroll-down to the Ingress Rules section.
This section shows the currently available security rules in the selected Default Security
List.
Next, click Add Ingress Rules.
In the Add Ingress Rules dialog box, add the rules for the TCP protocol which enables you
to use Cloudera Manager and Hue to access your cluster.
You can customize this dialog box to meet your security needs.
In our example, we are opening up the port to all sources by entering 0.0.0.0/0 for the
Source CIDR.
Anyone on the internet can access port 7183 for CM and port 8888 for Hue.
No egress rule is required to allow the response traffic.
Update the source to a more restrictive CDIR if required.
For the Source Type, select CIDR.
For the Source CIDR, enter 0.0.0.0/0.
For the IP Protocol, select TCP.
For the Source Port Range, enter ALL.
For the Destination Port Range, enter the port number that you'd like to use to access
the cluster such as 7183 for Cloudera Manager.
Enter an optional description.
Click +Additional Ingress Rule to add the second ingress rule for Hue.
At the bottom of the Add Ingress Rules dialog box, the Ingress Rule 2 section is displayed.
Enter the following details: For the Source Type, select CIDR.
For the Source CIDR, enter 0.0.0.0/0.
For the IP Protocol, select TCP.
For the Source Port Range, enter ALL.
For the Destination Port Range, enter 8888.
Enter an optional description, and then click Add Ingress Rules.
The two ingress rules are added to the Default Security List.
Let's start Cloudera Manager using our reserved public IP address which we are not showing
here for security reasons, followed by a colon, and port 7183.
On the CM Login screen, enter your username and password.
Remember, use the password that you specified when you created the cluster.
Next, click Sign In.
The CM Home page is displayed.
Note the name of the cluster, the CDH version that we chose, and the services running on
the cluster.
From the Hosts tab, click All Hosts.
The five hosts of our cluster are displayed.
Hosts are configured with one or more role instances, each of which corresponds to a
service.
The role indicates which daemon, also known as service, runs on the host.
Typically, a host will run multiple roles in support of the various services running
in the cluster.
Let's drill-down on the roles associated with the Master node in the cluster.
The services and roles that are running on the Master node are displayed such as the
HDFS NameNode, Spark History Server, YARN's Job History Server, and YARN's ResourceManager
among other services and gateways.
The node also runs some gateways.
A gateway is a type of role that typically provides client access to specific cluster
services.
For example, Hive and Spark each have gateway roles to provide access for their clients
to their respective services.
Click the Hosts tab, and then select Roles.
The Roles page displays a table that is grouped by hosts having the same roles assigned to
them.
For example, worker nodes 1 and 2 run the same roles such as the DataNode and NodeManager
services.
Let's start Hue using our reserved public IP address which we are not showing here for
security reasons, followed by a colon and port 8888.
The Hue Home page is displayed.
For information on how to use Hue, use the URL at the bottom of the screen.
The screen shows some of the useful BDS and OCI documentation resources.
Thanks for watching!
Learn more at: https://docs.oracle.com/en/cloud/paas/big-data-service/