Hi, everyone, and welcome to our session on what’s new in Google Search.
My name is John Mueller.
I work on the Search Relations team here at Google.
A part of my role is to connect those at Google, who work on Google Search,
with the folks out there making websites.
And that includes everyone who works on websites,
be they developers, site owners, analysts, marketers, perhaps even including you.
Our team’s goal is to give everyone the information they need
to allow the websites they work on to be successful in Google Search.
We'll be talking about SEO, but wait, SEO isn't a bad thing.
This session is specifically for developers,
so we’ll be focusing on the technical aspects.
We’ll go into recent updates that you can implement
as well as updates to the debugging and monitoring tools for Google Search.
Before we dig into the details, let's chat briefly about SEO.
Search Engine Optimization, or SEO in short,
is not some voodoo magic that you have to be worried about.
While SEO has a lot of non-technical aspects to it,
which are also important,
we'll be focusing purely on the technical side here.
Now, I can only speak for Google, but most of the basics
will apply to all search engines, and you’d run across similar challenges
whenever you want to fetch and parse webpages,
which is basically what search engines do.
A large part of technical SEO is about making it possible for search engines
to fetch your HTML pages and to understand the content that’s hosted there.
For that, they need to know about your URLs first.
Once the URLs are known, they can be fetched, which we call crawling.
Crawling is similar to how you might use wget or cURL to request the page.
After the page has been crawled and ideally rendered like a browser
to process any JavaScript, the results are then parsed.
We call this indexing.
Extracting headings and other texts from the DOM is kind of straightforward,
as is storing them as tokens for an index of your URLs.
Going past that, however, is hard.
How do search engines interpret that content?
Well, search engines use various machine-readable elements.
These elements can tell us which pages to store in our index
and which pages to skip, but there’s even more,
such as information that the current page is not a family saga but rather a pie recipe
and that the baking time is going to be 45 minutes.
We call these kinds of elements structured data,
and we'll see more of them in a bit.
With the processing, we also look for links to new pages,
links both to within your website and going out to other websites.
In order to understand the links, they should be well-formed HTML elements
and point at reasonable and fetchable URLs.
In short, we have the crawling of the pages, then comes a processing or indexing.
To complete the circle, we look for new URLs linked on those pages for future crawls
but also, we take all of this data and use it to generate the search results.
Those are the aspects I wanted to call out about technical SEO.
There's much more to it, of course.
If you’re working on a website that’s meant to be highly visible
and where it’s important that users can find it in search engines,
I strongly recommend you work with an SEO expert to discuss the project early on.
Sometimes there are challenges which you can work together to avoid.
For example, how to best deal with login pages.
Catching these early makes it easier to move forward quickly,
rather than having to redo things later on.
So, without any further interaction delay, let’s move on.
One of the recent changes in Google’s fetching of web pages
is the shift to enable HTTP/2 crawling.
HTTP/2 is the next major version of HTTP,
the protocol primarily used for transferring data on the internet.
We made this change to make crawling more efficient.
With HTTP/2, we can open a single TCP connection
and efficiently request multiple files in parallel.
This means Googlebot, our crawler, won’t need to spend
as much time crawling your server as before.
In SEO terms, this falls into the so-called crawl budget.
Crawl budget is a mix of how many URLs Google wants to crawl from your website,
the crawl demand, and how many your URLs our systems think your server
can handle without problem, the crawl capacity.
With HTTP/2 crawling, this gets a little bit more complicated
because a single connection can include multiple URLs.
Overall, we find that HTTP/2 crawling
gives our systems the ability to request more URLs
with a similar load on your servers.
The decision to crawl with HTTP/2 is based on whether the server supports it
and whether our systems determine there’s a possible efficiency gain.
You don’t have to do anything special
apart from enabling HTTPS and HTTP/2 support on your web server.
Now, since starting this experiment,
we’re excited to be able to share that we’re now crawling
more than half of all URLs with HTTP/2.
Using stream multiplexing and header compression,
the number of connections and the bandwidth have both gone down significantly.
These improvements help both our crawling as well as your website serving infrastructure.
Woohoo.
Moving on to structured data.
When it comes to machine readable information on a page,
Google primarily relies on structured data embedded in the HTML pages.
Over the past year, JSON-LD has become the most popular way
websites provide structured data.
All of Google's modern search specific metadata
can be provided through JSON-LD and the schema.org vocabulary.
Schema.org is a globally accepted standard & open vocabulary for expressing information.
It was started 10 years ago
and is now an important part of the open web infrastructure.
It provides a stable foundation on which websites can be built for the long run
and yet is constantly being expanded as new technologies and features are needed.
A shared vocabulary makes it easier for developers
to get the maximum benefit for their implementations
and enables others to use the data in appropriate ways.
For example, Google recently open sourced Schemarama.
Schemarama provides data parsing and validation tools,
which make it easier for you to consume schema.org data
as well as to integrate automated testing into your production pipelines.
If you’re curious about what you can do with structured within Google Search,
I’d recommend checking out the search gallery in our developer documentation.
Consider the type of website you’re building,
and then look for specific elements that would work well with it.
There’s recipe structured data, as I mentioned before,
but also newer options, such as for online and offline events,
options for home activities, and even practice problems
for those who are building a website for students.
Giving extra information like this won’t increase your website’s ranking in Search,
but allows your pages to present more visibly
in the search results when users are looking for them.
We also use structured data for videos.
One of the areas where we’ve been making changes over the past year
is how we recognize and show videos in Search.
In particular, there are a few kinds of supported structured data,
which can be quite interesting if embedding videos is important to your website.
Taking a step back, videos and their landing pages
are prominently shown in Search and Discover and are popular with users.
You can host videos either on your own website
or with many popular video hosting platforms.
Both approaches work similarly and are supported by Google Search.
In addition to helping users find videos, Search now allows users
to directly skip to important moments in a video,
making it quicker and easier for users to access your content.
To enable these features, we plan to support two types of markup.
First, there's clip markup.
With this structured data markup, a webpage can provide information
on segments or clips within a video.
These segments can then be shown directly in the search results,
making it easier for users to go right to a part of a video.
It doesn't take much to do this.
You just need to provide a textual tag, a starting time,
and a URL that goes directly to that timestamp.
Then Google Search will be able to show those videos appropriately,
but what if you can't easily list the segment information
for all of your hosted videos?
For example, perhaps many people are using your website
to host videos on their own.
Well, for these cases, you can soon use seek markup.
With this markup, Google can use machine learning to analyze your video content
and automatically determine relevant segments for you.
All you have to do is tell us how to link to an arbitrary timestamp
within the video hosted on your pages, and we’ll do the rest.
It's magical.
To enable this feature, use seek structured data on those pages
so that we know how to link to parts appropriately.
There's more about this feature and its current status
in our Search developer documentation.
At the time of recording, it's still in a pilot
but we hope to make it generally available in the near future.
All right, over to our tooling.
The primary set of tools we recommend for debugging
and monitoring websites for Search is aptly called Search Console.
If you haven't used Search Console before,
to get started, you need to verify ownership of the website you're working on.
If someone else is already working on Search for your website,
then it’s likely just a matter of having them add you to the site.
For more information on this setup,
I'd recommend the Search Console training video series.
Within Search Console, there’s some neat new features
that might make your lives a little bit easier.
First, there's crawl stats. We touched upon crawling earlier.
The new crawl stats report in Search Console gives you information on that.
For example, how many requests were made, what your servers were responses were,
and any availability issues along the way.
As a developer, you're probably used to having to analyze server log files
to get this kind of information.
Having it directly in Search Console makes it a bit easier for you
to recognize changes and to dig into issues.
Then there's a page experience report.
You’ve probably heard folks talk about the page experience ranking factor.
Search Console now has a report just for that.
It gives you a summary of the good URLs on your site
as well as information on the other parts of the page experience ranking factor.
We define good URLs as those which reached the good thresholds
of each of the Core Web Vitals.
As I mentioned, we also gave information
about the other parts of the ranking factor too,
such as HTTPS, mobile usability, and the safe browsing status.
This report is a great way to get started on page experience
and to spot broader issues which a developer might be able to fix on a website.
We have more information on this report, as well as detailed information
about the page experience update in other videos linked from here.
To be honest, it can be quite a complicated topic,
and there are many specifics where developers can help out.
It’s worthwhile to get a good understanding of the details.
All right, let's take a quick look back.
We talked briefly about SEO, and roughly how crawling and indexing works.
These aren't magical black boxes
and you can help to make those processes work better.
Then we touched upon crawling with HTTP/2
and how it can help with the load on your site servers,
both for users and Googlebots crawling.
Next, we took a look at structured data,
which is where many of the new features of Search come from.
In particular, markup which you can use for embedded videos.
And, last but not least, there’s Search Console,
the ever-evolving tool for debugging and monitoring a website for Search.
Make sure you're set up and have access to those reports.
There’s definitely a lot more that has changed in and around Search,
but I thought this collection would be particularly useful to you all.
In any case, I hope this session gave an insight into how things
have been evolving in Google Search.
We're definitely not done yet.
The world online is constantly changing,
as is the work that you all have been creating for the web.
We’re keen on making sure that Google Search remains relevant
for users, site owners, and developers.
If you’d like to stay in the loop for future updates,
the Search Central site and its YouTube channels
are great starting points.
With that, thank you for watching, and goodbye.