Every business professional from an SEO specialist to a stock or cryptocurrency forecast expert needs
a quick and cheap way to scrape Google. After all, the quality of their work depends on it. That's
why we are reviewing a promising and affordable Google Search Results Scraper - SERPMaster.
In this video we’ll give you a tutorial on how to scrape Google search results with SERPMaster,
and if you stick till the end, you’ll also get an overview of its pricing,
functions and user experience.
In case you're new to the topic, let’s start from the basics. SERP is an abbreviation
of the search engine results page, displayed after you enter a query into a Google search.
The websites you see on these pages aren't random or accidental. In fact,
they are ranked according to the keywords used and various other factors. Since most people select
the very first search result, often completely ignoring the rest, everyone wants to rank on top.
Knowing this, you can probably guess what SERPMaster is and what it does.
That's right. It's a specialized tool called SERP API designed to collect information from
search engines, be it organic results, ads, or product information. Having access to this data
allows SEO specialists to build their strategy of becoming more visible to potential clients.
SERPMaster works by downloading the search page and then extracting the right kind of data from
it. What sets it apart from a simple web scraper is that SERPMaster handles everything from proxy
management to CAPTCHAS and data parsing. It means that all you have to do is form the right
query by using some of your coding knowledge and send requests to the API. If you perform
this correctly, SERPMaster promises to deliver structured results with a 100% success rate.
By now you’re probably asking “How do I scrape a Google search page?” Well, here’s a quick tutorial:
We will show the simplest way to scrape data from a Google SERP page.
The very first step of starting to use SERPMaster is signing up. Once you do that, you'll receive
an email with the authentication information, which will provide you access to the SERP API.
As you probably can tell from our code, we are using cURL, a handy text-based tool
that runs in the terminal and allows transferring data over the internet.
First, you’ll need to enter your username and access token into the u parameter.
You’ll receive those after signing up.
Then it's time to specify q, which represents the query you will be making.
For this example, our search term is Hello world.
The last step of this query is to specify that you're sending application/json content.
Here's how the code should look like for the search term “Hello word".
Keep in mind that some systems, like ours, require escaping quotes with a backslash.
We've got results printed out in the terminal window. But you might not like the looks of that.
This is the raw HTML content. It's messy and hard to read. Don’t worry, though – SERPMaster
allows you to parse search results. Let's just set the parse parameter to true,
and SERPMaster will structure the scraped Google page. Now we've got our results in JSON format.
So that's how you use it.
If you see that SERPMaster is exactly what you were looking for, the next thing you
probably want to know is the price. Most scraping service providers charge high prices; as a result,
customers overspend and pay for the features they don't even require. While SERPMaster
offers budget friendly data acquisition, so you don’t have to worry about overspending.
Is that really true? Let's crunch some numbers.
So, SERPMaster offers a variety of plans but no option to pay as you go.
The smallest plan, Micro, starts from just $20 per month and gives you 5,000 requests.
This comes down to $4 for 1,000 results. The price per result is pretty high,
but entry plans are never efficient. If you have $92 per month to spare,
the price drops to $2.30 per 1,000 results. Now we’re talking.
Meanwhile, its biggest plan offers 100,000 results for $200
which is $2 for 1,000 results. Oh, and for those who want to
test things out first, SERPMaster offers a free trial with 250 results.
Now let's compare how its prices fare against the competition.
As you can see, SERPMaster sticks true to its claims.
You must be thinking, 'if it's so cheap, it has to cut corners in some way.'
Fair assumption, but surprisingly that's not really the case.
SERPMaster is pretty rich in features. Let's take a look at them.
First of all, SERPMaster can retrieve all data types from Google.
Seriously, here's the list to prove it:
Whether it's organic, paid, or twitter results, everything you normally see on
SERPs is there. Oh, and you can switch between mobile and desktop SERPs.
The targeting options are also impressive. The tool allows you to pick any country in the world.
But if you need a little more precise targeting,
it also offers state and city-level results. You can even specify coordinates and radius.
SERPMaster isn’t a standalone point-and-click tool, so you’ll need to integrate it. The tool
can handle three methods of integration: The first one is called Browser, where you
send requests by forming long and complex URLs straight in your web browser's address bar.
It’s best suited for testing the tool. The second is called Real-Time. Here,
you send POST requests to SERPMaster over an open connection and receive data in… well, real time.
The third method is called Callback. Once again, you send a POST request.
But instead of retrieving results immediately, SERPMaster lets you fetch them whenever you wish.
Usually, it's done via webhooks. This method works best when you constantly need large amounts
of data. However, you can't schedule searches, which is something SERPMaster's competitors offer.
With SERPMaster, you'll be able to select which search results page
to extract and how many entries it should include but up to 100.
All of this data will be returned in raw HTML or parsed JSON formats. By the way,
the JSON parser can handle most data types.
Its output is neat and well structured, so you likely won't need to clean up the data much.
Now that we've discussed the features, let's talk about what it's like to use SERPMaster.
To start using it, you'll have to fill in the form and pay, first. Then customer
support will send you the credentials which will allow you to access the SERPMaster API.
The process is the same for the free trial, except you don't have to pay for it.
When it comes to requests, SERPMaster accepts various parameters that allow
specifying a keyword, location, device, and more. You’ll need to write them by hand,
but it’s really not that hard, even if you have little coding experience.
There are multiple APIs for specific types of search data, though they don’t differ much.
However, if you get confused about making requests, specifying a query, or anything else,
the tool has detailed documentation, that includes a walkthrough guide.
The main drawback of this service is that there's
no dashboard or sandbox to play around with the tool.
Everything is performed either via API or customer support.
Speaking of SERPMaster's customer support, there's no live chat,
so don't expect an ultra-fast response. However, the agents work 24/7,
so they won't keep you waiting forever. Keep in mind that if you sign up for
an entry plan, you'll miss out on something SERPMaster calls personal support – basically,
calling your account manager directly instead of simply shooting emails
Now let's discuss performance. As we mentioned before,
SERPmaster claims to provide a 100% success rate. We tested it to find out whether that's true.
We sent 10 requests per second to various Google URLs for our test, which resulted in 100,000
requests in total. With a timeout of 30 seconds, 95.78% of the requests connected to SERPMaster's
API. Out of those requests, 100% successfully reached Google. This is a terrific result.
To sum up everything I've just said,
SERPMaster is a low-cost, fast, and relatively simple way to scrape google search results.
It has a variety of features and its customer support shouldn't let you down. Although there
are three things SERPMaster could include: CSV output, request scheduling and API playground.
All in all, SERPmaster does what it promises and does it well. If you need moderate amounts
of data from Google and don't want to build the web scraping infrastructure by yourself,
consider SERPMaster. It’s a great option.
I hope this video answered some of your questions about SERPMaster and helped you
get started using it. But if you still have questions, drop them in the comments below.
Oh, and if you need more information about SEO monitoring, check out this video here.
See you in the next video!