A couple of weeks ago I passed the Databricks exam, in today's short video I want to share
with you some of my experiences and tips on how to pass this exam
and if you're new to my channel, I'm Riz Ang and I build a data platform, first thing first
why would you take the Databricks exam well if you are a developer or working as a data engineer or
data scientist you might have come across a lot of certifications out there, whether it is Microsoft
certifications AWS certifications or Google Cloud certifications and Databricks which is
basically a curated Apache Spark also provides a number of certifications that you can take
now. The goal of taking these certifications is generally to make you more marketable,
marketable in a sense that if you're working in a job if you have some of these certifications, it
gives you an edge over your co-workers and if you're a consultant like me when we are
providing about to provide the service to our client will love us a lot more if they
know that we are certified and of course you can put this certification on your Linkedin profile
so that the world knows that you are certified, and you know this product or service all right.
First thing first, the exam that I took was the Databricks certified associate developer for spark
3.0, which is the latest version and I did it in python, if you want you can take it in Scala
as well and I think there's also another certification
the professional data scientist exam and that is available and there is another one coming soon
that's not available yet which is the professional data engineer Databricks certification I believe
it should come sometime this year. The exam has 60 multiple choice questions and you have two hours
to do it and you can take the or start the exam 10 minutes before the scheduled time, you need 70%
to pass i.e you need 42 questions answered correctly of 60. All right now on to my
tips on this, first of all it's more about before the exam so my first tip is if you can if you work
for a company that can pay for your for training with Databricks, I would suggest you do that
because it will help you a lot meaning, you get a Databricks instructor who can guide you
in architecture as well as the practical questions that you may have while you actually coding spark.
Now I myself took a training I believe it's called Databricks cohort training,
which is really helpful for me and there were also some self-service training as well that i did
which was again useful for my preparation,
if you can't afford or if for some reason you can't get training, read books,
the book that helped me a lot is this Learning Spark, chapter one three to seven are the ones
that you want to focus on and there's another one called Beginning Apache Spark with Databricks
and for both books I will share you the link in my in the description below for you to check it out
okay and also if you want you can also test Databricks with Databricks community
edition, which again I will link the the link down below in the description.
My third tip is to familiarise yourself with the spark documentation,
the reason is because you actually get this documentation during the exam so you can actually
read this documentation to help you pass the exam, the problem is it's not as intuitive as googling,
there's no search button and where you basically want to search what function does what,
so you actually get a pdf file and I suggest you familiarise yourself with the spark documentation
and iIll share with you the documentation in the link down below in my description again.
My tip number four is this exam is online proctoring meaning there will be
actually no one that will monitor you or wait or sit with you making sure you're not doing anything
that you're not supposed to do but you, basically have to stay in front of your laptop or computer
with the camera on all the time if you need to go to the bathroom please do it before the exam
because for two hours you basically can't do that you have to stay in front of your laptop for two
hours, now also for some reason because of the online proctoring the moment you start the exam
you will only be able to work with your main screen and in my case I have a laptop and extended
monitor and for some reason I just can't use my extended monitor so I just end up with my main
screen my laptop which is quite tiny unfortunately I don't like it, but I had to live with it
and again coming back to my point number three the spark documentation is not,
you have to scroll down a lot and because this is in my laptop and the the laptop the the exam
screen has the questions in it and also the spark documentation on the side it's very small
to actually read the spark documentation so it's been useless really the documentation.
My next tip is make sure you familiarize with the spark architecture, this is crucial you will get
a lot of questions about this and what I mean by architecture is how spark engine works and
you have to understand what's the concept around the cluster, what's underneath the cluster the
driver and the worker concept, what are the nodes whether the driver node or the worker nodes
work, and within each node there's an executor which is like a space where the task gets done
and also a core or slot or thread what it's called, where this is basically
the machine that does the job. Now there's also application that is broken down by jobs,
a job broken down by stages and stage broken down into tasks, just make sure you understand all this
because you'll get a question about it make sure you understand, what's the difference between
narrow versus wide transformation and there is a such a thing called action versus
transformation make sure you check that out because it will be asked, just for information
transformation is lazy nothing is actually done or action with transformation whereas action is where
data gets actually read and processed transform and written etc, you need to make sure you
understand these differences, shuffling where data gets shuffled across different
nodes, repartition coalesce is again different way to change the partition so the data within
the partition itself bucketing and garbage collection, I think the garbage collection
it might not come up but I think it's in scope so make sure you aware of what that is. And
the next one is obviously Dataframe makes you understand well what Dataframe is use it
code it actually test it how to read and write from different file formats from CSV,
JSON, parquet file even Delta, and how to write a schema or get a schema out of your DataFrame,
printing it creating a temporary local view and also caching or persisting your
data frame and also make sure you understand the difference between caching and persisting,
all this is only a clickable button from google or spark documentation but if you want to know
more about it feel free to ask me in the link down below I'm happy to try to answer it.
My last tip about understanding the the exam content or makes you prepare for the exam
is makes you practice all of these spark functions, now this is not all
of them but this is probably some of the common ones that I used or I tried myself
so again I mentioned about action which is basically what gets action by spark
engine, things like collect, count, first, have, take, and show get familiarise on what these are,
different type of kind of transformations, distinct, drop, duplicates, filtering so on and
so forth, until where, untyped transformations, aggregate, apply, and all this transformation make
sure you aware what what these are and how you use it, it's more about how you use it because
in the exam you actually get questions about there's a scenario to do something, and you
have to evaluate the PySpark or the Spark code that's been written and you need to make sure you
understand and to understand that well you need to code it you need to type it yourself, joins,
it's pretty straightforward left in a right joint, aggregate functions, again counting, approximate,
count distinct, even all the way to variance and standard deviation, explode collection
,registering user define function, date and time function so on in a unix timestamp, non-aggregate
function like coalesce, column or literal or lit, sorting functions and string functions as well,
this is sort of a list of things that I had and I need to make sure that I tried every single one of
these before the exam and familiarised with them and that helped me a great deal to pass the exam.
At the end of the exam after you submit your results you will get your unofficial
outcome immediately whether you pass or you fail when you pass congratulations you will get
within usually three to five days an email from Databricks that you've now achieved the Databricks
certification and they'll give you a badge something like this one.
That's all for today's video I hope it's useful like or subscribe to my channel if you find
or if you like content like this and that's it thank you for watching I'll see you next time.