Hello and welcome, Today I’m going to show you how to create robots.txt file and also sitemap.xml file for seo purposes.
Best way for me to demonstrate how to create these two particular files is to give you an example. I’ve got www.rankya.com.au open now if I was to simply type
robots.txt file, you will see that it is simply a text file with some user agent directives. Now as far as we are concerned, we need to put this particular information into this file, because Google will honor this
particular directive, now if we didn’t have this directive, then Googlebot will not honor it. What this is saying is to Googlebot, saying “ok when you come visit my
website, I will disallow you to index these particular folders” now you can have files, you can disallow some file, as in some html files, php files and so on, or you can also, disallow some folders.
Now why would you want to do that? Is because, let’s say that you have a privacy policy file, or you got some terms and conditions file on your website, which most websites do have, now I don’t really want to
rank for these two particular files, you know they are important for my customers, but for ranking purposes, for seo purposes, they are not that important for Google, they are important for my potential
customers, but once again we are in the business of Google ranking, therefore, we need to be intelligent, as to what we want to rank for, and what we want to disallow Google to rank us for and so on.
Therefore we need to have this directive targeting Googlebot specifically, if not, Googlebot will simply index all your pages and so on. Another important element of robots.txt file is the URL for your
sitemap, that’s why we will talk a little about sitemap file, how it is important and also we’ll look at how to create proper sitemap files. There are many online generators for creating sitemaps, but I will give
you as usually, I will give you important knowledge for creating sitemaps. Therefore you need to have a sitemap; it is intelligent for your Google rankings to have a sitemap, if you haven’t got one, simply
go online, do bit of research and generate your xml file, and then simply upload it on to your root folder of your website, ok? But this is saying, let’s go step by step, when Googlebot comes and
visits your url, what it will do is, it will first look for this file, and when it does, it’s going to say “has it got any information for me? hmm, it has” therefore, it will honor this, right? But also, since it is still reading
it, it’s going to say “hmm there is a sitemap here” see? and then it is aware of that url therefore it will then visit that particular url. So let me now quickly open that particular file, to show you, what this is all
about right? Now, this particular file contains some information in the form of xml, and what I’m saying here is “I have a url which is my main url, it has been modified at this date, it gets changed weekly, and it
has a priority setting set as 1, now that’s the highest that I can have, surely this is my home page, therefore, it is intelligent to have my home page as the most important page, therefore, I say it in that
priority setting. Ok? now that’s kind of important, and also, a lot of people still make an honest mistake, as to, they think that if they were to have that information as daily because which you can, they think it
is intelligent for them, now, the only time you should have that changefreq set to daily, is if you change you content daily, and if you don’t change it daily, and if you change it weekly, then you should have
it weekly, and if you change it monthly, then you should have it set to monthly. It’s that simple.
Also, as you can see there is another page, surely rankya.com.au at this stage is a small website; you may have hundreds of pages, nonetheless, sitemaps are important for you to create now, once
again, when we look at this url set here, we need to have a look, we need to be smart here, when we look at this, we need to think “hmm ok, my home page is the most important, but the way that I
structured rankya.com.au, as far as I’m concerned, this particular page is the second most important” therefore, I set that priority, as to point 9, that’s my most important page most important resource,
and this is my second most important resource ok? And this is how we do it we say it here ok? And also, this particular url, I’m saying well then that one comes as far
as priority settings are concerned. And if scroll down you will see that the way I structured it, is I’m going from 1 point 0 to lower ok? so that kind of important, as you can see I do not have my contact page here,
as url set, now there is nothing wrong having that, but we are in the game of ranking, therefore we need to optimize these particular files in such way, that it serves their purpose, ok?
And this is a, perhaps you know, quick example as to what these pages are all about, we need to be intelligent as far as setting these, a lot of sitemap generators you will come across, they will simply have
information as to, it will have 0 point 5, and that’s like neutral it’s like baseline it’s like middle line, ok? Therefore it’s up to you to then go ahead and start thinking a little saying “ok I have this website” surely
all your pages are important to you, but, you know once again, you are in the ranking game, right? Therefore, you then got to sit down and think “hmm, ok, I know that all my pages are important, but, which
ones are more important than the others?” because when you start thinking like that, you’ll end up coming with some that you will set higher priority than the others, ok? So that’s very important.
And once again, you can simply Google “robots.txt Google webmasters” to find out little bit more information about this particular file, and also, you can visit, sitemaps.org website and read
more about it, because these are important elements for Google ranking, surely they are not the only part of seo, nonetheless, having these simple procedures set properly, will only benefit your Google
rankings, ok? Also you may come across information as to, telling you these particular files are not important. I promise you and I assure you, do not listen to misinformation like that, because they are
important. I hope you have enjoyed watching this particular video, and if you would like to find out more about search engine optimisation methods,
simply visit us at rankya.com.au thank you very much