For those curious to know
how Google is combating spam, there’s a website “How Search Work”. Matt Cutts
has released a video on How Search Workswebsite, letting people know the
interesting areas within the site they should be navigating to as well as what all
Google has been doing to improve the search.
In the video, Cutts says
that How Search Works is a small site that talks about advanced steps Google
has taken in crawling, algorithms, and how it fights spam.
Even if you have seen the
site before, Cutts wants you to walk you through some of the things that you
might not have seen and are quiet nice. So, here are the details of what Cutts
has revealed about the site in the video.
If you go to the main page
of the site and scroll through a little bit, you will find almost an
infographic, which is actually interactive. You can click around and find all
kinds of fun easter eggs. If you get down the bottom of the page, it will tell
you that you have been on the page for 150 seconds. In that time Google has
actually handled 5.7 million searches or something along those lines.
As you look through the
site, Matt says that you will find Google talking about how it does evaluation.
There are videos on how Google evaluates search quality. Google do evaluate new
search algorithms and send them out to quality raters and they look at one set
of results on left and one set of results on right. They have to decide which
one looks better. They don't know what algorithms have been evaluated. And when
they vote, Google takes the results and figure out which search results got
better and which ones got worse. But, Google doesn't directly apply those
ratings into ranking algorithm.
Google actually shows the
funnel for the things in recent year. In 2012, Google went through 118,000
ideas where it played around with new way of generating search results and
using the ratings it got from quality raters, it was able to say that this
looks like a promising experiment. From there, it did 10,000 side by sides,
where you get side-by-side set of search results and ask people which one you
like better. Based on that, Google did 7000 live traffic experiment where it
actually take an experiment and put it on the main website and look how often
people click at actual search results to try to determine it was actually
making the search results better. Matt also said that as a result, Google was
able to 665 algorithmic changes on search result page in 2012. If Google puts
that into context, it's roughly two changes to how it generates search engine
pages every single day for the entire year.
People come and ask what
happened on such and such day because there's lot of stuff happening because
there's usually lot of stuff happening. Things rolling out, new data being
deployed, and those are actual changes and not the data being refreshed. So, it
gives you a little bit of feel for the scale of how many different changes were
exploring at any given point.
The part of How Search
Works, Matt Cutts enjoys the most is the spam section. Google goes into all
kind of information, you might not have seen before. For example, there's a
spam carousel and that is updated periodically. So, you actually get to see
spam right after Google has removed it. A screenshot is showed so that you
don't get infected by malware or something. It's literally like you can watch
over Google's shoulder as it is removing spam. So, you get a chance to see the
sort of stuff Google has to deal with every single day.
Below the spam carousel, the
site has different types of spam; the categories of spam -cloaking or sneaky
redirects, hacked sites, keyword stuffing, hidden text, pure spam, and parked
domains. Others include spamming free hosts or dynamic DNS providers, thin
content with little added value, unnatural links from a site, unnatural links
to a site, user-generated spam. There are more specific, more granular, and
more detailed things within each one of those. So, unnatural links from a site
might involve someone who is selling links that pass page rank. But, that gives
an idea of the overall categories Google look at whenever, it is fighting spam.
Google also gives different
graphs to tell month by month the actions taken of taken on spam. The vast
majority of what Google tackles is what it classifies as pure spam or black hat
spam. It means, the gibberish stuff that anybody would be able to recognize if
they are sufficiently savvy.
Another thing that you would
notice is the next biggest category within recent years has been hacked site.
In 2010, some SEOs wrote "what has the web spam team been doing, we
haven't seen a lot of action from them recently". Google was at that time
engaged in a hand-to-hand combat on hacked sites, which a regular SEO or a
black hat SEO might not have noticed as much.
You can get these kinds of
insights when you look through these graphs and the history of the sort of
stuff that Google has had to tackle in terms of spam.
Google has also started to
do more messaging over time. It can do probably better to get more concrete and
actionable messages to webmasters. If you look at the milestones of what Google
has done, it is pretty exciting as you can see volume spike up as Google has
started to get more and more information. At this point, for pretty much any
direct action that you take, that the manual webspam team takes that affects
you ranking, the webmaster will get a message about that. That's really helpful
because you know there's an issue and you can start to deal with it and dig
into it and start to investigate a little bit.
In January 2013, Google sent
over 431,000 messages as a result of actions that it took on the webspam team.
So the other thing you should think about is the scale at which Google is
operating. These are all manual webspam actions, which ten generated some sort
of message to the webmaster. The idea that Google can have one-on-one
conversation with 431,000 different owners of websites sort of shows the scale
that Google is operating at and why it's hard and why so far it hasn't figured
out a way to have a one-on-one conversation with every single webmaster who
wants to rank number one or rank highly or has questions about potential
webspam action.
Below this information, is a
graph that shows the reconsideration requests that have been submitted. And so
for a random week in 2013, there were roughly 5,000 reconsideration requests
and over a month 430,000 messages go out. If you turn this week long base line
into a month, about 20,000 reconsideration request processing messages is what
Google handles during a month. What is interesting is that out of all the
people that Google sends messages about manual webspam action being taken, only
5 percent request reconsideration. So, most of the time Google is killing spam,
spammers decide to move on to try to do it on a different URL. It's kind of
neat to take some of these numbers, compare them, and play a little bit with
realizing what insights we can get from these kinds of graphs. It shows you the
scale of the problem. If you have 20,000 people a month who want to talk to you
why they think their website should rank highly when
Google thinks that it has
at least violated the guidelines, you see the sort of difficulties Google has
in trying to talk to everybody. Google will keep trying to do better, be more
transparent, but it's fantastic that Google has got How Search Works website
and some dashboards where you can see how things are going. You can see live
examples of spam, as they get thrown out. Cutts thinks that you will really
enjoy the website.
