In this course, I'll be providing a very quick overview of GCP,
in particular, it's foundational parts,
the compute and storage,
and then we'll dive straight in to provide
a deeper overview of the different ways that you can process data with GCP.
So, we'll talk about BigQuery, Dataflow,
Dataproc, Cloud SQL, Datalab, et cetera.
So, who's this class meant for?
It's meant for what we call data engineers.
People who design, build,
maintain data structures, databases,
data processing systems, data pipelines;
who do extraction, transformation,
loading of data, move data from one place to another;
maybe data scientists who are analyzing data,
enabling machine learning to happen,
maybe even doing machine learning yourself;
people who model business processes,
who enabled data-driven decision-making within your company.
So, if you recognize yourself in any of these things,
this class is meant for you,
it's meant for anyone who'll be working with data on Google Cloud Platform.
It's also meant for any decision makers who are trying to decide
whether your company should move to GCP and do your data processing on GCP.
So, this gives you a good overview
of all the capabilities that the Google Cloud provides,
so that you can make an informed decision.
In the course, we will start with an overview of Google Cloud Platform as a
whole but with particular emphasis on the ways that you can handle data in the platform,
the way you can ingest data,
different ways of doing processing of data whether with MapReduce,
such as with Spark or whether with a streaming mechanism like with dataflow,
we look at BigQuery which is our auto-scaling data warehouse et cetera.
So we'll give you an overview of GCP,
but with an emphasis on the data handling parts of it.
We'll also then move on to talking about the foundation of GCP
and this is computing and storage like any computer and the Cloud is a computer.
The two key parts of a computer are the computing units and the storage units,
a persistent data, and the way that happens on
the Cloud is with Compute Engine and Cloud Storage.
So we'll talk about both of those and then we'll move on
to things you're probably doing today,
that you could easily move to the Cloud.
We'll talk about the use cases that are quite common,
that Google provides a good managed environment for,
such that you can take things that you're doing on premise,
on your own hardware and move it to the Cloud quite easily because
the same software that you may be using is also present on the Cloud.
So as examples of those use cases,
we will talk about relational databases in
the form of Cloud SQL which is a MySQL database,
hosted on GCP and we'll talk about Cloud Dataproc,
which is a hosted version of PySpark all the Hadoop ecosystem software processes.
So, in order to do that we'll look at how to import data and how to
query MySQL running in Google Cloud and similarly,
we'll also look at how to take a Spark program submit it and have it run on
Cloud Dataproc and the Spark program that we will look at will
be a machine-learning program to carry out recommendations.
Once we have talked about the use cases that we're talking about migrating use cases,
we will then move on to talking about more transformational use cases.
These may be things that you may not be doing today,
mainly because it may not be possible for you to do them today.
So, for example we look at how you could query petabytes of
data in a matter of seconds with Google BigQuery,
we will talk about how to do fast random access,
trading of global consistency versus no global availability,
we'll also talk about machine-learning,
we'll talk about how to do TensorFlow,
maybe you run TensorFlow but very commonly maybe running it on a single machine.
We'll talk about how you'd run TensorFlow in
a distributed fashion on the Cloud over extremely large datasets.
Then we will move on to just providing a very quick overview of how
you would do scalable reliable data processing on Google Cloud with Cloud pops up,
which is a messaging architecture,
and with Cloud Dataflow,
which is a way to execute a code
that processes both streaming data and batch data in essentially the same way.
Finally, we'll come to a conclusion to the course and I'll leave you
with some resources for further reading.