In many PhD disciplines, students need to generate some
data sets to perform some analysis in order to get answers
to their research questions or to get an evidence for the
hypothesis or whatever is their point of view. In most cases,
these data sets are generated by using surveys. However, a large
amount of data sets are already available on government
websites, which students can use freely, because in most cases,
they have a Creative Commons license. Almost all developed
countries have these types of websites where data sets are
available, and some of the developing countries also have
these websites. In this video, I'm going to show you some of
the websites for Australia and us where you can get free data
sets to do some analysis for your research. And stick around
till the end of this video, because at the end, I will show
you some of the fastest growing occupations not based on my own
opinion, but based on the data, which we will get from our
US government website.
And we are starting right after this short break.
Hello, and welcome to my channel. My name is Tahir And if
this is your first time, please hit the subscribe button and
also the notification bell to get notified whenever I
post a new video on this channel
every Tuesday and Friday. So before I show you the data sets
websites, it is important for you to understand few things.
First of all, most organizations they are working on the fear
principles of data, and what are fear principles of data. In this
fear principle F stands for pointable ever accessible I for
inter operable and are for reusable. So I want to explain
these terms and few more before I show you the data set websites
because some of you might be interested in knowing how this
works. So when we say data should be finable, it means that
these data sets must have some DUI or digital object identifier
similar to general papers or articles, they have a DUI as
well. But using this DUI, anyone can find this data set and use
it and when we say accessible, it not necessarily means that
all data should be open to everyone because there are some
privacy and security concerns. And some government
organizations may restrict the access of the data. But if you
know that data is available, you can get hold of that data
by using some of their procedures.
And in Australia, all of the government agencies, they have
to put all non sensitive data as open data on government website,
and I'm going to show you in a minute. And when we say
interoperable, it means that the formats we are which we are
going to use should be a community agreed format. For
example, some data sets have CSV format, some has Excel format,
or PDF format, or some other JSON or other formats. So all
these formats are acceptable community format, so everybody
can use it on various operating systems. And when we say really
usable, so it means that this data must have some metadata,
which links it to some contextual information. And we
must understand what are the context in which this data was
collected. And here I want to mention a very important thing
is that sometimes students are struggling to find all we can
make some questions for our questionnaire. The good thing is
that some of this data also shows you some of the questions
for your research. So what are the good questions for a
particular data which have been used by previous researchers, so
you can follow their path and you can select some of the
questions for your own research and use them in your own
questionnaires. I have a separate video about
questionnaires which you can find on my channel, how to make
a research question is something like that you can search and you
can find so this was the fear principle, and no one further
detail I would like to mention here for ODI, there is an
organization which is called do.org. And if you look at their
website, there is a tab for registration agencies and you
should be aware of these agencies because these are the
agencies which a lot are which register this digital object
identifier for any data set. Another thing which I would like
to mention here is that in this video I am going to show you
websites where you can get data mostly for social sciences where
you can get data for population geography, economics and these
kind of things. For technical are for engineering data sets. I
will make a separate video where I will discuss about machine
learning repository UCI er for exemple are reddish, there is a
robotics repository. So if some of you are engineering students,
please let me know and put something in the comments about
your discipline, and I will make a separate video for engineering
students. Another thing is if for example, you are looking for
some data for your own specific country, not for us or
Australia, you should search for similar websites for your own
country. And definitely you will get some websites from where you
will be able to
get some data.
No, I will jump on my computer. And I will show you the largest
government websites of Australia and us. The first website, which
I would like to show you is Australian government website
data.gov.au. This is the largest source of open data from
Australian government. So these are free data sets available
here. And anyone can access this data. And this data has a lot of
value for growing the economy of the country. And this is the
purpose, why government agencies put the data on these websites
so that everyone would be able to contribute to the economy of
the country. And if you click on this data sets, you have a lot
of filtering options here. As you can see that if I click for
example, any location, I can select any particular state of
Australia, I also have the ability to select any of these
formats available HTML PDF format. And then there are some
Excel formats available and TIFF format for imaging and some
other formats are also available. So very easy to use
website. And you can also select any dates from 1492 to today's
date, so you can just put the starting year here and the last
year in this column. So very intuitive website. And I hope
that you would be able to use these public datasets. And the
next website is Australian National Data Service. And this
website is up is developed by four partners. And if you click
on this about us and about us home, these are the four
institutions Monash University, Australian National University,
Commonwealth Scientific and Industrial Research
Organization, and Australian government or national research
infrastructure for Australia. And here they describe the core
purpose of this organization to make Australia's research data
assets more valuable for researchers, research
institutions, and the nation. And to explore this website,
your starting point should be disturbed working with data. And
these are the failed data principles, which I have
mentioned earlier, you can read about them further. And then you
click on these guides and resources and guides and
resources homepage, and you will find a section getting started.
So this should be your starting point. And I hope again, that as
a PhD research student, you should be able to explore this
website and get some useful data for your research. And this is
an exemple data from Australian National University here, you
have to get an account with them. But anyway, I just wanted
to show you the digital object identifier, this is the value of
DUI for this data set. And if you have this value, you can
search it on the internet. And this data has a lot of
information. Before getting the data files, you can have a look
at whether this data will be useful for you or your project
or not. And then you can create an account and get these files.
The next website is Australian Bureau of Statistics abs.gov.au.
And this website also has a lot of statistical data. So if you
click on the statistics, now we'll come to this page. And as
you can see that economy data, data about population health
data, industrial data, labor data and a lot of other data
types also available, including the micro data. And you can also
have some guidelines about methodology and standards. And
here, you can browse by using the catalog number once you have
these, or you can also browse by release date if you have some
clue about the period of the data. So another very useful
website, I encourage you to explore this website as well,
no, probably the largest data website for us is census.gov.
And this is a department which does not only count the
population of United States, they also have a huge amount of
data for us. And in order to access that data you need to
type a sub domain which is data.census.gov.
And here you can read about this website if you want further
information. Otherwise if you scroll down, you have all these
data are available. There is also an advanced search option,
which you can access by clicking here. So let me click on this
population data. And let's say we want this employment data.
And as you can see that there are a lot of tables 683 tables
here, if I click on this, now this table is available. So in
order to download this data, you have to click on Customize
table, then there is a download link. If you click here, then
you can select if multiple years are available, sometimes you
will see a lot of ears here. But this data is only for 2017. And
the format available is only CSV PDF is not clickable. So PDF is
not available for this data. And then if you click on download,
you can simply download this data. So just click on download
now and it will be downloaded. So I just cancel this. So this
is the largest website in us. And know at the end, I would
like to show you Bureau of Labor Statistics. This is the bls.gov.
This is also us official website. And there is some
interesting data available here for you. So if I click on this
data tools, you have various data retrieval options here,
customized tables, and some of their publications are very
useful. And one of those is this Occupational Outlook Handbook.
If I click on that, you can have some very useful information as
a student so you can see what are the fastest growing
occupations. And if I click here, I will see this chart and
this shows that this occupation is having a growth rate of 63%.
If you are working in any closely associated field, you
might think going to this profession later on once you
finish your studies. And here you can have all these
occupations which are growing fast, and you might select one
for yourself. And as you can see that this data is quite recent.
So there are a large number of other websites available where
you can get public data sets for free. But this was just the
first introductory video and some of you will start using
some data sets and if you have any questions, please put in the
comments. And I will make some more videos on this topic later
on. So I hope that you like this video, if so please give it a
thumbs up and subscribe to my channel. Thanks for watching.
See you next time.