I have heard some chatter that the results have reverted back, however almost two weeks later I am surprised to say that the geo problems in the UK SERPS still exist. The UK search results are as bad as I have seen them, they are quite simply shocking in terms relevancy for a UK audience. The Google.com results from the UK (IP) have always been more of a half way house between UK/US results and now these results seem to be even more US focused aswell.

I recently asked Matt to comment, but he choose not to respond which is fair enough. It’s difficult to know the cause of this change, a tweak in the geo algo or some have suggested problems with Googles link graph due to a massive UK ISP hack. Lets take a look at some of the results where we can highlight the problem that still exists.

A search for ‘web hosting’ on Google UK brings back a number of US sites in the top 10. Webhostinggeeks.com which compares a load of US web hosting companies with all prices in dollars is 7th. Justhost.com is 8th which may look like a UK site with a quick glance (if you look from the UK as it’s delivering price/phone number by IP) but you can see from Googles cache its very much a US site in their eyes. Webhostingstuff.com again another US site takes up the 10th spot on the 1st page.

The 2nd page of results is much worse and really shows the scale of the issue.

Webhostingsearch.com, Webhosting.info, Findmyhosting.com, Webhostingrally.com, Hostseeq.com, Envisionwebhosting.com and Webhostingratings.com are all US sites now ranking in the UK results.

So out of the top 20 results for ‘web hosting’, 10 of them are US based sites with US prices. That’s 50% of the results with a US bias. Even more scary is that every single one of the sites on the 2nd page is basically an affiliate comparison site.

Sure some visitors from the UK will be willing to pay on their credit card in dollars for a (sometimes) cheaper web host as often people don’t care where it’s hosted.  You could argue (badly) that Google has decided that for this query, UK searchers habits have shown that US sites derserve their positioning as it’s still relevant? Ok, so lets look at another example for ‘flowers‘ -

The first page results are actually not to bad with a lot of UK results. But the 2nd page again is where the problems are obvious. In 15th position there is 800florals.com and just below 1800flowers.com. These are both US sites, with US prices which only deliver within The States and Canada. What exactly are they doing in the UK results? At the same time well known relevant online UK florists such as Arenaflowers, Flyingflowers, Debenhams and Bunches sit below them.

A search for ‘SEO’ on Google.co.uk now pulls back SEO.com a US based SEO agency in 8th position. On the 2nd page we now have Aaron Walls Seobook.com and SEOmoz.org. Both great sites in their own right with good content, but they have appeared from nowhere knocking UK based content further down the listings. It’s not just US sites either, there are some examples of Australian sites ranking higher (not as high as US sites) for generic phrases or for longer tail, less competitive queries.

So the questions are, is this a test? Were these changes made on purpose and are Google working on the issue?  The answers would be no, no and hopefully yes.

While Webmasters and Google UK users are scratching their heads at the poor SERPS, Bings results are looking comparatively much better.

I think some negative matching might be required for Bingo PPC advertisers. :-)

I noticed this quite sometime back when Google started crawling and indexing new blog pages within minutes and ranking this content very quickly in their search results.

Crawling and indexing content so quickly was developed to support their query deserves freshness (QDF) algorithm which enables Google to show hot new content almost immediately for the freshest and most up to date search results.

However what I noticed is that while blog pages can be indexed and ranking within minutes, a couple of days later the blog post is nowhere to be seen anymore. It disappears. The page is still in Googles cache, but when you either search for the url or the exact page title the post will no longer come back under search.

So I performed a little test with my last post to highlight this phenomenon.

• 17th March – The ‘Linkfromdomain Command – Much underused‘ post was written and published. It was subsequently crawled and indexed that day and showing in the SERPs for url and blog post title searches.
• 18th March – Page still showing in SERPs when searching for url and blog post title.
• 19th March – Page still showing in SERPs when searching for url and blog post title.

• 20th March – Page cannot be found in SERPs when searching for url or blog post title. The page is still in Google cache though.
• 21st March – Page cannot be found in SERPs when searching for url or blog post title.
• 22nd March – Page cannot be found in SERPs when searching for url or blog post title.

• 23rd March – Page can now be found again in the SERPs when searching for url and blog post title.

The results above show the page was showing in the SERPs for around 3 days before disappearing for around 3 days. Looking at the crawling behaviour of Googlebot it visited the page on the 17th 4 times, before visiting again on the 18th and has since not returned.

So Why Is This Happening?

From my experience this is certainly not this blog alone where this happens although it’s hard to say for certain whether this happens to every blog. Is it something to do with domain authority or links? I have seen content both trusted and new sites with both few and lots of links fall inline with this behaviour. I have seen this on at least half a dozen other sites for example. Does this happen with news articles which are crawled and indexed quickly in the same way? Possibly.

My theory is that Google either takes time to migrate the minty fresh index content into their usual index or after the initial bump in the QDF algo Google takes time to re-evaulate & intergrate the page into their usual algorithm before showing it in the SERPS again. I would be interested to hear Matts comments.

Is This Important?

It’s frustrating to have content disappear from the results obviously as you can lose traffic. But especially if you want to take advantage of Googles QDF algo or have content in Googles index at a certain date, you need to strategically time your publishing of posts with this in mind. Or you could find that the planned content you wanted to rank will not show just when you need it!

1) Check Your Outbound Links – You can’t always control who links to you, but you can control who you link out to which means it’s a fantastic indicator of trust to the search engines. We have seen time and time again when sites start linking out to poor neighbourhoods or spam (hacked sites or the DP network as extreme examples) your rankings die. So use the linkfromdomain command to see if you are linking out to anything you shouldn’t be.

You can use the excellent free Xenu link sleuth for this but with the linkfromdomain query you can add the usual spam queries to the end to narrow your search down.

For example for this blog with viagra. They all seem fine, although Yahoo still has some of it’s old hacked pages in the cache.

If you want to find out exactly what page you are linking to the site from use this command on Yahoo.

2) Identify Authority Hubs – Ok, so we believe that Googles trust algo was built around an initial hand picking of ‘tier 1′ authoritative trusted sites initially. The more links you receive from sites like these and the ‘closer’ you are to these sites, the more authority and trust you will have in the search engines eyes. It can be extremely difficult to get links from these sites, so identify who they link out to and get links there.

For example, the BBC. Then you can narrow your query down, for example if you wanted to find out who the BBC links out to with content around ‘flowers‘.

3) A little Naughty – How about as a way to find old domains with established pagerank / links? Using the BBC as an example again, how about expired domains or domains for sale which you can look to pick up. But obviously you can identify the authority sites in your niche or sector and analyse those sites.

So are you still using linkfromdomain?

Even for their own sites.

Google have since this morning stopped this from working, but the basis was to check pagerank on the domain of your choice with hyves as the subdomain.

You will now get one of three different results for the pagerank of this non-existent subdomain -> PR7, PR4 and PR0.

PR7 means that the domain is perfectly fine
PR4 means that the domain got penalized by google
PR0 means that the domain got banned in Google

Sounds crazy right?  Well I was pretty skeptical until I tried it on a number of domains that I know that do and don’t have Google penalties. I know some have mentioned a few discrepancies, but all those I checked supported the conclusion above entirely. Weird.

To add a little more fuel to the fire, I thought I would run a mass pagerank check on subdomains of a number of directories. Now, we know Google started penalising low quality directories a while back, removed directories from their recommended guidelines and then put them back in again.

So before Google took away the ability to use the hyves trick I quickly scanned a load of directories on the old strongest directories list by Aviva. Old I know, but I wanted something to mass test. The results are as below.

Directories Not Penalised (Pr7)

Page Strength 10

http://www.dmoz.org
http://dir.yahoo.com

Page Strength 9.5

http://www.business.com

Page Strength 8.5

http://www.botw.org
http://bubl.ac.uk

Page Strength 8

Directory.v7n.com

Page Strength 7.5

http://www.ezilon.com

Page Strength 7

http://www.familyfriendlysites.com
http://www.goguides.com

Page Strength 6.5

http://www.bloggeries.com
http://www.dirjournal.com
http://www.elib.org
http://www.gimpsy.com
http://www.joeant.com
http://www.maxlinks.org
http://www.skaffe.com
http://www.tygo.com/dir

Page Strength 6

http://www.allinfodir.com
http://blogannounce.info
http://www.cascandra.com
http://www.dmegs.com
http://www.incrawler.com
http://www.kahuki.com
http://www.onlinewide.com
http://www.prolinkdirectory.com
http://www.rakcha.com
http://www.top5jamaica.com
http://www.ventedoy.com
http://www.webahead.net

Page Strength 5.5

http://www.anthonyparsons.com
http://www.blazemp.com/dir
http://www.ebusiness-directory.com
http://www.enquira.com
http://www.global-weblinks.com
http://www.linkwith.us
http://www.nzpages.co.nz
http://www.pegasusdirectory.com
http://www.seoma.net
http://www.web1directory.com
http://www.webotopia.org
http://www.zorg-directory.com

Page Strength 5

http://www.123kidzarea.com
http://www.alikedirectory.com
http://www.gbguide.com
http://www.homesalez.com/directory
http://www.linkspub.com
http://www.massivelinks.com/
http://www.postdotcom.com
http://www.qoobe.org
http://www.umdum.com
http://www.web10.ws

Directories Penalised (Pr4)

Page Strength 7

http://www.avivadirectory.com
http://www.romow.com

Page Strength 6.5

http://www.2yi.net
http://www.index-it.net
http://www.mygreencorner.com
http://www.relmaxtop.com
http://www.webworldindex.com

Page Strength 6

http://www.allydirectory.com
http://www.busybits.com
http://www.browse8.com
http://www.domaining.in
http://www.ebjuris.com
http://www.gii.in
http://www.sevenseek.com
http://www.site-sift.com
http://www.szab.net
http://www.uksmallbusinessdirectory.co.uk
http://www.wowdirectory.com

Page Strength 5.5

http://9ug.com
http://www.abilogic.com
http://www.apahcinc.org
http://ask-dir.com
http://www.authoritydirectory.com
http://blogaboutmysite.com/directory
http://blogtagstic.com
http://www.cannylink.com
http://www.dirwizard.com
http://www.emillie.net
http://www.e-topic.com/directory
http://www.idk.in
http://www.kwikgoblin.com
http://www.linkopedia.com
http://www.linksjuice.com
http://www.makeasearch.com
http://www.map100.com
http://www.nobledirectory.com
http://www.photarium.com
http://www.qango.com
http://www.vxbox.com

Page Strength 5

http://www.alivedirectory.com
http://www.businessseek.biz
http://www.ensuredirectory.com
http://www.ewebpages.org
http://www.excellentguide.com
http://www.fullofsearch.com
http://www.linkcentre.com
http://www.maxdirectory.eu

Directories Banned (Pr0)

http://www.uncoverthenet.com

So What Can We Takeaway From This Data?

First of all, it’s worth noting I haven’t had a chance to go through all the above and verify the results. Or that all the sites are working, what their current Pr is or whether they actually have a hyves subdomain etc. :)

But at a top level view, I think there is a clear difference in quality between the sets of directories which supports the hyves theory. There are a few low quality directories in the ‘not penalised’ group, but perhaps these simply got through the net.

If my memory serves me correctly uncoverthenet.com was banned from Google completely quite sometime ago and it still seems to be the case based on the above results. Their full domain is also a Pr0 and not showing for their own name.

Obviously don’t take this data at face value – I am not saying you should go and submit your site to any of the non penalised directories, far from it. Directory submissions are very low value these days, this was purely for analysis before Google took it away.

Key Points From Hyves -

• It’s pretty easy to spot a site that has been penalised already. So this data didn’t really tell me much more than you could already work out yourself from a quick analysis.
• There was not a lot of consistency on sites penalised – Some sites selling links but not others, some sites involved with text link ads but not others, some on the DP network, but not others etc. Again, like we already know!
• Another point to remember is that SOME of the pagerank penalisations were toolbar only at a superficial level and links from these sites may still have value.

Anything I missed?

Broad -   golf clubs
Phrase – “golf clubs”
Exact -   [golf clubs]

Now I haven’t heard or read anything official from Google personally, but a member over at Webmaster World gives the Google representatives reasoning as -

“Instead of triples of all keywords, they want advertisers to go from “broad” to “narrow” (in their words).”

Although Google may have a point that every keyword does not need to be replicated for each match type it is suprising to hear (if true) because actually you should start completely the opposite way with your PPC campaigns. So ignore Google on this one and I will explain in more detail why. If you don’t already know what each match type does, read up here.

Start Narrow

Don’t waste your money by using the default broad match straight away. Start narrow with specific exact, phrase and negative keyword combinations before even thinking about using broad match.

You can start with all exact match phrase keywords before building out to phrase match, but if you are confident and use negative keywords properly phrase match is easy to use. Not all keywords will require an exact and phrase match equivalent either, but absolutely those keywords with medium to high volume should have an exact match version. Equally those lower volume keywords might not require a exact match version. So to take the golf clubs example, I might start with only an exact match version of [golf clubs] as it’s very high volume and could potentially appear against an even larger number of terms if on phrase. However, a lower volume term like “buy golf clubs uk” probably wouldn’t require an exact match version and you could start this on phrase match straight away.

This approach means you do not need a different match type for every keyword and helps to keep your campaigns managble when dealing with hundreds of thousands of keywords already. I know some agencies and bid management software companies recommend replicating keywords into each match type which is fine, but you will often find that your campaign is bloated and a large part of it is unnecessary.

The narrow to broad approach also applies to the actual keywords you are using aswell as the match type, so use specific keywords with long tail variations, rather than more generic general phrases to begin with.

Make sure your campaign is performing with these match types to start with – monitor your internal logs and run search phrase reports (to see what people really searched for) to expand your current keywords or add further negatives to your campaign.

Use Broad Match To Sweep Up

If you have additional budget or want more volume you can then think about introducing broad match, although the expanded match element does need to be very closely monitored.

Use broad match to ’sweep up’ any low volume keyword variations that exact and phrase match types have not already covered. When using broad match, try initially going in with a lower bid than your exact and phrase match equivalents and it can produce great results. You will need to continue to monitor your logs and run search phrase reports to further expand your keywords (& negatives). Again, aswell as match type you can also start to trial more generic search phrases for further volume (on exact match of course, have you been listening?).

Splitting Adgroups Based On Match Type

Not always, but sometimes it makes sense to split up adgroups via match type. You know when using an exact match phrase that for the advert to appear that [exact phrase] must be searched for, so it makes sense to focus your advert on this term specifically. When using broad match, your advert might appear against a number of keyword variations so sometimes it makes sense to diversify your ads a little more and test. You might find that you just make a new adgroup for just one or a few of your exact match keywords as you need greater control over them for example. Remember, trial, review and test what produces the best CTR and more importantly conversion for your campaign.

Optimise Match Types Based On Performance

Obviously the great thing about using various match types is that each match type will perform differently from the other. So each should have their own separate bid and be optimised based on their own individual performance. This allows you to have greater control over your campaign and ultimately spend money where it makes the most sense for improved return.

Note – Don’t forget, the choice of match type does not impact your quality score. All match types for the same keyword have the same quality score regardless, so this is not a factor. They do obviously massively influence your CTR though, which is the biggest factor of Googles quality score algo.

Normally Google do not charge for clicks on new advert testing like this which is good news. But for me it’s quite scary how far they are taking the expansion of where they show their adverts.

At the top of search suggest is far to easy for accidental clicks in my view although Google are also testing showing ads at the bottom which would seem better. Another concern for navigational brand searches, are Google offering you any real value in showing your advert here? Especially with increased use of Chrome which combines a traditional url bar with search.

Google recently removed keyword trademarking in the UK and subsequently profit through increased brand bidding from advertisers in fear of losing brand traffic to competitors.  In many cases advertisers do not need to bid against their brand or the money would be better spent elsewhere, so will these adverts fuel even more fear? Johanna Wright Director of Product Management at Google argues -

“This is another example of search becoming more dynamic and Google getting users to the correct results as fast as possible,”

Is Google really focusing on the user experience here or is this a revenue making exercise? Lets hope Google make this opt in or at least have an opt out feature at the time of its (potential) introduction, not months after.

Google had already quietly changed their policy at the end of October allowing PPC ads for beer. However according to the Inside Adwords blog the new rules are for promotion and branding rather than direct sale which are still not allowed -

“To comply with the updated hard alcohol and liqueurs policy, advertisers must promote the information about hard alcohol and liqueurs that their websites contain, such as recipes and brand messages. Ads that directly promote the sale of hard alcohol and liqueurs are still not permissible through our program. In contrast, advertisements for beer may directly promote its sale.“

The news comes after Google firstly removed their keyword trademark policy in the UK and then permitted advertising for online gambling in the UK, Italy, France and Spain.

Obviously only the cynics would correlate this with an attempt to subdue the effects of the current economic climate, Googles subsequent fall in share price and predicted fall in ad spend for 2009 where 97% of Googles revenue is still driven from. Although it is certainly good to see Google making decisions on a local level meaning big opportunities have opened up for those businesses (and in turn search marketing agencies) in these industries.

Google were obviously equally unhappy with the search results as I was last night. In my opinion they were as poor as they have been in the last few years. A lot of big, big sites homepages went missing from the search results completely and much for the worse. I put this down to a bad data push rather than an algo change, but its hard to say. It certainly wasn’t obvious link buyers or sellers (or network schemes) who were targeted in the industries I know, it was far more random and unpredictable. In some cases non secure (http) versions of homepages were replaced with secure (https) versions even when the secure version had zero internal/external links. Strange behaviour indeed.

I actually managed to catch a screen shot of the ‘PPC blog‘ SERPS early yesterday evening. Considering this site ranks no.1 in Google UK for this term (has done since its creation) with sitelinks, it was quite strange to see it drop all of a sudden to 5th place. Not only that, as you can see there were 3 results on the 1st page, 2 of which were indented and strangely one being the homepage.

I could theorise as to what Google were attempting to do if this was an update, but I really don’t need to. This site is as white hat as it comes, all editorial links with zero link purchases/selling or exchanges and just good old fashioned unique content. So if this site ranks badly in the SERP’s, Google is quite probably going in the wrong direction. In my opinion of course :)

More seriously, there were multi million pound businesses just dropping out of the search results completely in some verticals I monitor. Not just one or two, but a number of big players in each industry. Is this a sign of things to come? No, as the changes were rolled back. But there is something obviously brewing.