For those of us that have been around the search engine indexing change saga, that happens about every 6 months to a year or so, seeing Google change the way it indexes and ranks data is a regular occurrence. Sometimes with few visible effects - sometimes with devastating effects. In the SEO industry, we're so used to Google rolling out algorithm changes without fanfare and reacting to them as we realize something has shifted that this announcement came as quite a surprise to me. Thank you Google for the warning this time!  The announcement is that Google is about to launch it's most aggressive change in quite some time with a new data storage system and more importantly the Google Caffeine engine update. So what is up with this new Google Caffeine upate and how will it effect your particlular websites.

Addressed here is the Google Caffeine search engine update and what affects &/or changes it may have on website rankings compared to the old system of ranking. What kind of differences can we expect. Will the earth move under our feet regarding your website search engine rankings when Google rolls out Caffeine or will the affect be nearly invisible?

{UPDATED 11-24-09}

A Little Background

As with most search engine companies, including Google, they keep their methods close to the vest when it comes to just how they index and rank websites and pages. As for search engine optimization we can only observe and test to determine what methods and techniques we can incorporate into a website to effect it's search engine ranking position.

In past years there was much that could be done to affect a websites placing in the search indexes through proper keyword phrase targeting, title, description and ketword meta tags, along with the most important: good original keyword rich content. Much of this was often abused by webmasters through keyword stuffing. Through a series of algorithm changes over the years Google has de-emphasised most of these methods paying less attention to meta tags and boosted their focus on the actual content and backlinks from authority websites.

Once Google went public back in 2004 and the corporate bean counters took over we have seen a greater lean towards Google optimizing monitization in advertising. Contrary to the Google apologists denial there is an observable effect in a websites ranking and what the website owners payout to Google in adwords advertising. There has been much debate from both camps arguements on this issue.  Personally, I have seen a bumping in the page position on numerous websites once they begin to engage in more aggressive Google advertising. I state this from my own observations and at the risk of Googles displeasure being waged on this website in the way getting a spanking in my page ranking. 

{Edit: 09/01/09 - Here's Matt Cutts comments, from Google, about this issue on Web ProNews, basically saying YES, links from Google properties does have an effect on page rank and uses Google's Knol as an example. "Sometimes we do flow pagerank, but we try to be relatively careful because we're mindful of the people who do try to abuse it," he says.}

The net effect of this course toward increased monitization of the search engine/advertising industry, particularly the emphasis on backlinks (which can be bought), is it helps to enable those with the fattest wallet to get the best ranking. The more you can afford to spend on incoming links and advertizing to your web site from what Google considers "authority" websites, the higher your ranking.  This opens a huge door for businesses to "buy-their-way" to the top of the search engine rankings regardless of the quality of their content, product or service.

So What's up?

The new search infrastructure, Google Caffeine, is the beginning of Google's advance towards improving indexing speed and scale as the size of the web grows increasingly large. Google is also seeking feedback on the changes from experienced users and web developers by making Caffeine available via a web developer preview. According to Matt Cutts of Google,  Caffeine is essentially a rewrite of the search index and it roughly compares with the Big Daddy index of late 2005 / 2006. In other words, it's a BIG change to Google search. "We're shooting to get results identical to previous version. We'll open up a few datacenters with it first and then roll it out.  Caffeine will be more powerful, flexible and robust - allowing Google to index faster," Cutts states in a video on his blog. "(Caffeine) builds a powerful foundation for including any changes we want to do with indexing. Not so much for taking advantage of semantic, real-time indexing, but for getting good infrastructure in place for growth and unlock more power."

The new Google Caffeine update is looking like Google still has their sails set in the same direction as past updates with a few small observable differences which I will outline here later. Google gave us all a sneak peak by opening up their Caffeine engine on several sandbox test servers for webmasters and website owners to test. ( http://www2.sandbox.google.com/ ) This prompted several others to set up websites to be able to do comparative analysis, side by side, of the Pre-Caffeine engine verses the soon to be launched Caffeine engine, in a kind of decaffeinated vs caffenated taste test. Use this link, run a query and see how the two search engine results compare line by line and site by site: http://www.facesaerch.com/caffeine/

From Google's official blog:   Google states, "It's the first step in a process that will let us push the envelope on size, indexing speed, accuracy, comprehensiveness and other dimensions. The new infrastructure sits 'under the hood' of Google's search engine, which means that most users won't notice a difference in search results. But web developers and power searchers might notice a few differences."

A Caffeinated and Decaffeinated Google Taste Test!

After running numerous queries for the keyword phrases for the websites I deal with here at Beach-net, here is a few of the differences I observed:

• Probably the biggest surprise for me was the marked jump in keyword density between SERPS (search engine results page) on the old Google and SERPS on the new Google Caffeine. In nearly every comparison, the Caffeine SERPS featured site titles and text copy or description showed a much higher keyword phrase and/or keyword density. Coincidence? I doubt it.
  
  
• After using a backlink checker tool on many of the websites at the top of the indexes there is definitely a strong emphasis on backlinks or incoming links, quantity & quality, as there was before, particularly from what Google considers "authority" websites (not sure what criteria Google uses to determine authority).  This trend continues to widen the door for websites to "buy their way to the top" of the indexes through purchased one way links.  This is much to the dissadvantage for those with a more limited budget.
  

• I also noticed a large drop in ranking of what I call "aggregator" web sites.  Some have seemed to dissapear from the top 3 - 5 pages altogether. Aggregator websites are those websites that appear rather high in the search engine rankings that are, to put it bluntly, just another index. I always find them irritating when I'm searching for something because the aggregator sites linked on that index, when clicked, give me nothing more then another index of sites related to my query.  Ummm... but excuse me - wasn't that what I was using Google for in the first place? It irritates me and I'm sure others, to use an index, then to click a link on that index only to get another index. I'm sure it irritates and confuses others as well sending them through numerous pages and clicks before they get to the content they were looking for in the first place. So, devaluing these so called "aggregator web sites" I think is a good move.
  
  
• Longevity or domain age seems to have a bit of authority as to just where the sites are ranking. Websites that I have known that have been around for a long time and continuously updated their content over the websites lifetime seem to have a little better edge in the rankings both with old Google and the new Caffeine.
  

• The actual placement of websites on the page between the two were very similar.  Some web sites dropped a bit where as others rose to a higher position.  Generally, after some analysis, this had to do with amount of content, keyword density, quality and quantity of backlinks and the presence of a blog associalted to the website.

• Caffeine is definitely faster. Every search query I tried on Caffeine was returned at a faster speed than with the current Google. Impressive.

• Caffeine seems slightly fresher. Some of the results I observed in Caffeine SERPS and not in regular Google SERPS were more current. For example, blog posts published within the last couple of days show up.

• Apart from the ego search, old Google out performed Caffeine in the index size category. But this is likely because only a handful of data-centers have Caffeine on board so far.

• I've observed a greater emphasis on social networking web sites such as sites like Blogger, LinkedIn, Facebook and Google Profiles featuring more prominently, particularly for name searches. Wiki pages still seem to rank highly in both Caffeinated and Decaffeinated Google.

Over all, the single most important factor in the rankings of the websites I have tested,  on both Google versions, is without a doubt backlinks.  This has been very noticable in the old pre-Caffeine Google searches and is even more noticable in the Caffeine test servers.

All in all it's still way to early to predict exactly what the effect on rankings for a particular website will be as Google admits the Caffeine data centers are fewer in number and the engine is still in beta testing as they are looking for feedback for future tweaking.  I'll be keeping an eye on this issue as Google prepares an official launch for this update.  When Google indicates they are moving closer to a roll out of their Caffeine update I'll keep you posted here on this website with what I see.

{UPDATE 11-24-09 } "I know that webmasters can get anxious around this time of year, so I wanted to reassure site owners that the full Caffeine roll out will happen after the holidays," says Cutts on his blog. "Caffeine will go live at one data center so that we can continue to collect data and improve the technology, but I don’t expect Caffeine to go live at additional data centers until after the holidays are over. Most searchers wouldn’t immediately notice any changes with Caffeine, but going slowly not only gives us time to collect feedback and improve, but will also minimize the stress on webmasters during the holidays."

The announcement at what used to be the Caffeine sandbox (test site) reads:

"We appreciate all the feedback from people who searched on our Caffeine sandbox.

Based on the success we've seen, we believe Caffeine is ready for a larger audience. Soon we will activate Caffeine more widely, beginning with one data center. This sandbox is no longer necessary and has been retired, but we appreciate the testing and positive input that webmasters and publishers have given. "