R is an open-source language and environment for statistical computing and graphics.
Oracle Big Data Manager provides a Zeppelin Notebook environment that lets you use Spark
R against a variety of big data and relational data sources, including IMPALA tables, R dataframes,
Spark dataframes, Oracle Database, HDFS and Hive tables.
Oracle R Advanced Analytics for Hadoop (ORAAH) is required when interacting with both Oracle
Database and supported big data formats in a single R session.
We’re going to analyze data in the open source IRIS data set.
This data set is commonly used to teach analytic and data visualization techniques in R.
It contains three plant species: setosa, virginica, versicolor, and has four features measured
for each sample.
For example, here is a Frequency Histogram of the Sepal Length and Sepal Width for each
of the plant species.
We’ve stored a copy of the IRIS data set in each of the supported big data storage
formats to show how each input format is accessed by R using Oracle Big Data Manager.
To begin, we load the Oracle R Advanced Analytics for Hadoop packages using the core ORCH Java
library.
Although not required, we create a custom function, named zepShow().
This function lets us easily identify each of the supported big data formats for the
IRIS data set.
So, let’s walk through the function definition.
Initially, the function verifies whether or not the ORCH packages are loaded.
If so, the first input format check is made to see if it’s a Spark dataframe.
In this code segment a random sample of the data is generated and the data is returned.
If the input is not a Spark dataframe, the function continues on to identify the input
format.
So, we scroll down to view code segments for the other supported input formats.
If the input data is HDFS, we simply construct a display format for the data set and return
the data.
If the input format is HIVE, we traverse the data, create a HIVE view with a random sample
and return the results.
Otherwise, we continue to check for additional input formats.
If the input is an IMPALA table, we pull the data, format it, and return the data set.
If the input is an R dataframe, the data set is traversed, a random sample is generated,
and the results are returned.
If not, none of the supported input formats have been used, and an invalid input error
is returned stating that case.
At the end of the zepShow() function, a final else clause handles the case that the ORCH
(ORAAH) libraries are not loaded.
In this case, only R dataframes can be used as input.
So, the dataframe is traversed, formatted for display, and the data set is returned.
Otherwise, an invalid input error is returned, stating that the ORCH library is not loaded,
and the input format is not an R dataframe.
Now, let’s look at the Zeppelin Notebook data visualization features.
The Notebook provides built-in tools that let you view your analytic results in these
formats: Table, Bar Chart, Pie Chart, Area Chart, Line Chart and Scatter Chart.
In addition, you can format the output in a variety of ways using the Settings option.
Here is the graphical output of the IRIS data set from a local R dataframe using the zepShow()
function.
We have run a random sample of the data, and select a Scatter Chart graph for visualization
purposes.
You can select any other display type from the graphical output toolbar at any time.
As mentioned, you can modify the layout of any graph by clicking Settings.
For a Scatter Chart, you can select fields for xAxis, yAxis, group and size options.
You can also select any data point to view numeric details.
Here, we select a Setosa species data point that shows its Sepal length and width.
Next, we show an example of connecting to HIVE from R using ORAAH.
First, we connect a specified HIVE host name with the ore.connect() function.
Then, we then check to see if the IRIS_HIVE table exists.
If not, we push the IRIS dataframe to HIVE as a table.
Then, we show the available tables in the HIVE repository with the ore.ls() function
and see that IRIS_HIVE table is there.
Now, using the zepShow() function, we view a Bar graph of a random sample of data in
the IRIS_HIVE table.
Here, the results include 10 rows.
But the random sample is generated from the entire data set.
So, if the table has a million rows, we get the same distribution with our 10 rows as
is found in the entire million row source.
In this example, we use ORAAH to connect to IMPALA from R using the same ore.connect()
function.
And, as before, we ensure that the desired impala table exists for our analysis.
Then, we use zepShow()to view a Stacked Bar graph of data in the IRIS_IMPALA table.
In this case, the IMPALA input is running a Top N Records result rather than a random
sample, that we used in the last example.
Next, we view graphical output from an HDFS file.
In this case we simply push the local R IRIS dataframe from memory to a temporary HDFS
file, which is mapped as an ORAAH DFS ID.
Then, we use zepShow()to view a Pie graph of the data.
Here, we are only running a Top N result, since HDFS doesn’t have analytic capabilities
such as randomizing.
As with any graphical output, click a data point to view its supporting data.
In this case, we see a sum of the Setosa Sepal Length values.
And, if you want to view the underlying input data for any graph, simply click the Table
tool in the display options menu.
You now know how to execute statistical analysis and visualize the results with R, against
a variety of big data input sources, by using Oracle Big Data Manager’s Zeppelin Notebook.
Learn more at cloud.oracle.com/bigdata