Although the basic layout is similar, the new design has a lot more space between each result, which will (in my opinion at least) mean an increased CTR on the very top positions.

You can see a full size screenshoot here.

I personally think this makes the organic results look pretty cluttered and I found them harder to read.

I captured a screenshot of a query for ‘flowers’ here in Google.co.uk. Notice how the words ‘Flower’ and ‘Flowers’ are capitalised in display URLs. Like serenataFlowers.com, Flowersdirect.co.uk, asda-Flowers.co.uk etc.

Almost immediately after I ran this query, Google stopped the test and I couldn’t replicate.

Notice the spaces in the display urls where the keyphrases within the search query are matched against the keywords within the domain in the display url.

So ‘superbreak.com’ becomes ‘super break.com’ with a space.

In my opinion this looks odd and is more confusing at a glance. This is obviously something Google are testing and alongside their recent background colour change to yellow, we have no doubt it’s with a view to increasing CTR.

I first noticed this about a week ago while running some queries in the US and the tests still don’t seem to be completely widespread just yet.

We first saw a yellow ad background colour in 2007 (before turning to a pink PPC background in May 2010), although it’s difficult to tell whether this is the same yellow Google is trialing again now. The yellow now in these tests looks a little paler.

We know Google tests different background colours from time to time to test click through rates (CTR) of ads with a view to increasing them.

Many have argued in the past the colour change is to attract more attention to the top PPC ad spots and hence a higher CTR. However, is the real reason behind the continued use of very pale colours actually having the opposite impact blurring the line between organic and paid results which is utlimately leading to higher CTR?

You would have to say so.

Update – We checked the new Adwords background colour and it’s coming out as (#FFF8E7) known as ‘cosmic latte’. Wikipedia says this is the same colour as ‘all light in the universe added up‘ and describes it as a slightly beigeish white. So….. beigeish white, very close to the organic white. :-) Although we tested the old yellow background and it does seem to be the same colour.

It has been a little while in development, but I am pleased to announce our first SEO tool for release, the Screaming Frog SEO Spider. I am a big fan of the Xenu Link Sleuth tool for finding broken links, redirects and other site issues, but wanted to build a site spider with more emphasis on SEO. I also wanted something that was usable for all SEOs that you can point and click at anytime and didn’t have to log into an interface. The ability to export everything to Excel was a must as I often like to map out SEO recommendations in this format.

What Is The Screaming Frog SEO Spider?

The Screaming Frog SEO Spider is Java based program which spiders websites links, images, CSS, script and apps. It also fetches key onsite page elements for SEO, presents them in tabs by type and allows you to filter for common SEO issues, or slice and dice the data how you see fit by exporting and importing into Excel. You can view, analyse and filter the information as it’s gathered and updated continuously in the programs user interface.

The Screaming Frog SEO Spider allows you to quickly analyse or review a site from an onsite SEO perspective. It’s particulary good for analysing large sites where manually checking every page would be extremely labour intensive (or impossible!) and where you can easily miss a redirect, meta refresh or duplicate page issue.

The spider allows you to export key onsite SEO elements (url, page title, meta descriptions, headings etc) to Excel so it can easily be used as a base to make SEO recommendations from. The following video should give you a better idea -

What Information Does The Screaming Frog SEO Spider Report?

A quick summary of some of the data collected -

• Errors – Client & server errors (4XX, 5XX)
• Redirects – (3XX, permanent or temporary)
• External Links – All followed links and their subsequent status codes
• URI Issues – Non ASCII characters, underscores, uppercase characters, dynamic uris, over 115 characters
• Duplicate Pages – Hash value / MD5checksums lookup for duplicate pages
• Page Title – Missing, duplicate, over 70 characters, same as h1, multiple
• Meta Description – Missing, duplicate, over 156 characters, multiple
• Meta Keyword – Mainly for reference as it’s only (barely) used by Yahoo the last time I checked. Missing, duplicate, multiple
• H1 – Missing, duplicate, over 70 characters, multiple
• H2 – Missing, duplicate, over 70 characters, multiple
• Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, noodp, noydir etc
• Meta Refresh – Including target page and time delay
• Canonical link element
• File Size
• Page depth level
• Inlinks – All pages linking to a URI
• Outlinks – All pages a URI links out to
• Anchor Text – All link text. Alt text from images with links
• Follow & Nofollow – At link level (true/false)
• Images – All URIs with the image link & all images from a given page. Images over 100kb, missing alt text, alt text over 100 characters

Enough Reading? Download Now

The lite version of the tool is completely free and allows you to crawl up to 500 URIs. The configuration options for crawling and excluding of file paths are not included.

A license for £99 per annum will remove the 500 URI crawl limit and allow access to the spider options menu.

What Is Next For The Screaming Frog SEO Tool?

The tool is a very early version and we have some big plans over the coming months with many new features in the pipeline. If there is anything specific you think we should add, please do leave the feedback below, send an e-mail or tweet it to me @screamingfrog. Although, looking at the list of features to add, it might well already be on there!

Keep Updated

Keep updated with future release of the tool by checking for updates in the interface, subscribing to the PPC Blog RSS, signing up to our e-mail above or following me on Twitter @screamingfrog.

But check out this test I spotted earlier today -

You can see a much bigger screenshot here.

As you can see, it allows Google to continue to have the maximum of three advertisers in the top spots for pay per click while still promoting their credit card comparison product aggressively. All at the expense of the algorithmic results, which get knocked right down the page…

Now it looks like testing time again, I am connecting to a DC with a pink/purple shade background -

And also a much much brighter quite striking shade of blue -

Are you seeing any other colours?

However, there is still a geo problem in the Google UK SERPs which is more noticable on the longer tail queries (as you might expect). Higher authority domains that don’t target the UK, still manage to overcome the geo filtering and outrank UK targeted websites with more relevant content which from a user experience perspective is simply put, poor.

As SEOs we find ourselves running search queries a lot for clients, but my experience today was as a normal user wanting to buy a product. I am after a Diesel man bag to replace the one I managed to break recently. Hence, I searched for ‘Diesel messenger bags’ -

So lets look at the results that Google.co.uk gave me (without personalisation) -

1. Ebay.co.uk – Ok, cool. But I want to buy brand new and don’t fancy browsing Ebay to look through reputable dealers.
2. About.com – Not really after a content farm.
3. Zappos.com – Doesn’t deliver to the UK!
4. Ebay.com – US Ebay site. I just decided not to buy from the UK site so…
5. Bizrate.com – Don’t really want to buy from a shopping comparison site but I’ll take a look. Oh wait there, it’s the US version anyway.
6. Nextag.com – Same as above.
7. Shopstyle.com – It has handbags in the url and is the US shop…
8. Askmen.com – A blog post, no help as I want to buy, buy, buy!
9. Shopping.Yahoo.com – Diesel handbags for the US market. Great.
10. Rushfaster.com.au – An Aussie site. Got to have an .com.au ranking in the top 10 of Google UK SERPS these days.
11. Ebay.co.uk – Oh FFS, Ebay and their bloody subdomains.
12. About.com – The content farm AGAIN….!!
13. Purseblog.com – AHHH another blog.
14. Thebaglady.tv – At least this snippet is telling me I can buy them at ASOS but its just a product review with aff link!
15. Nextag.com – Seriously, again?
16. Amazon.co.uk – WA-HEY! Finally a UK site that sells Diesel messenger bags! Oh, it’s out of stock though.

Now at this point you feel like shooting yourself in the face right?

Try and refrain from doing that and run the search again. This time, click on the PPC results. I found what I was looking for with the 1st click. The Adwords team must be loving the revenue right now.

There has been a fair amount of movement for the search query ‘search engine optimisation‘ in the Google UK SERPs recently. I’ll let you perform your own analysis and work out what’s going on if you haven’t already.

But what can we learn from the state of the results?

1) The power of exact match domains – searchengineoptimisation.org has been ranking consistently in 3rd. searchengineoptimization.co.uk the US spelling is 2nd page. The only real question here, is why Click Consult are not utilising searchengineoptimisation.co.uk which is parked.

2) Link networks, link buying, sponsoring website themes (for wordpress etc) and utilising your own clients to rank for SEO all still work.

3) Hiding links (from users) using CSS is difficult for the search engines to discover.

4) There is still far to much weight placed on anchor text and it makes the SERPS easy to manipulate.

5) SEO hasn’t moved on as far as many believe.