This is something I have been meaning to discuss for a longtime and is not something that I have really seen anyone talk about much.

I noticed this quite sometime back when Google started crawling and indexing new blog pages within minutes and ranking this content very quickly in their search results.

Crawling and indexing content so quickly was developed to support their query deserves freshness (QDF) algorithm which enables Google to show hot new content almost immediately for the freshest and most up to date search results.

However what I noticed is that while blog pages can be indexed and ranking within minutes, a couple of days later the blog post is nowhere to be seen anymore. It disappears. The page is still in Googles cache, but when you either search for the url or the exact page title the post will no longer come back under search.

So I performed a little test with my last post to highlight this phenomenon.

• 17th March – The ‘Linkfromdomain Command – Much underused‘ post was written and published. It was subsequently crawled and indexed that day and showing in the SERPs for url and blog post title searches.
• 18th March – Page still showing in SERPs when searching for url and blog post title.
• 19th March – Page still showing in SERPs when searching for url and blog post title.

• 20th March – Page cannot be found in SERPs when searching for url or blog post title. The page is still in Google cache though.
• 21st March – Page cannot be found in SERPs when searching for url or blog post title.
• 22nd March – Page cannot be found in SERPs when searching for url or blog post title.

• 23rd March – Page can now be found again in the SERPs when searching for url and blog post title.

The results above show the page was showing in the SERPs for around 3 days before disappearing for around 3 days. Looking at the crawling behaviour of Googlebot it visited the page on the 17th 4 times, before visiting again on the 18th and has since not returned.

So Why Is This Happening?

From my experience this is certainly not this blog alone where this happens although it’s hard to say for certain whether this happens to every blog. Is it something to do with domain authority or links? I have seen content both trusted and new sites with both few and lots of links fall inline with this behaviour. I have seen this on at least half a dozen other sites for example. Does this happen with news articles which are crawled and indexed quickly in the same way? Possibly.

My theory is that Google either takes time to migrate the minty fresh index content into their usual index or after the initial bump in the QDF algo Google takes time to re-evaulate & intergrate the page into their usual algorithm before showing it in the SERPS again. I would be interested to hear Matts comments.

Is This Important?

It’s frustrating to have content disappear from the results obviously as you can lose traffic. But especially if you want to take advantage of Googles QDF algo or have content in Googles index at a certain date, you need to strategically time your publishing of posts with this in mind. Or you could find that the planned content you wanted to rank will not show just when you need it!

I still love the linkfromdomain command on Live.com. Like MSN / Live (perhaps Kumo?!) search platform, it’s often forgotten about. But actually it still has much value for SEO. Here are 3 quick reasons why -

1) Check Your Outbound Links – You can’t always control who links to you, but you can control who you link out to which means it’s a fantastic indicator of trust to the search engines. We have seen time and time again when sites start linking out to poor neighbourhoods or spam (hacked sites or the DP network as extreme examples) your rankings die. So use the linkfromdomain command to see if you are linking out to anything you shouldn’t be.

You can use the excellent free Xenu link sleuth for this but with the linkfromdomain query you can add the usual spam queries to the end to narrow your search down.

For example for this blog with viagra. They all seem fine, although Yahoo still has some of it’s old hacked pages in the cache.

If you want to find out exactly what page you are linking to the site from use this command on Yahoo.

2) Identify Authority Hubs – Ok, so we believe that Googles trust algo was built around an initial hand picking of ‘tier 1′ authoritative trusted sites initially. The more links you receive from sites like these and the ‘closer’ you are to these sites, the more authority and trust you will have in the search engines eyes. It can be extremely difficult to get links from these sites, so identify who they link out to and get links there.

For example, the BBC. Then you can narrow your query down, for example if you wanted to find out who the BBC links out to with content around ‘flowers‘.

3) A little Naughty – How about as a way to find old domains with established pagerank / links? Using the BBC as an example again, how about expired domains or domains for sale which you can look to pick up. But obviously you can identify the authority sites in your niche or sector and analyse those sites.

So are you still using linkfromdomain?

Looks as though Google has a bit of a problem today because it’s labeling every site with the ‘this site may harm your computer‘ warning accross all it’s SERPS. This seems to be on Google.co.uk and Google.com. Oops.

Even for their own sites.

So if you missed it yesterday Mediadonis posted about a very strange subdomain trick that he and others claim allowed anyone to check if a domain was penalised in someway by Google (from selling links or loss of pagerank from other ‘malicious’ means).

Google have since this morning stopped this from working, but the basis was to check pagerank on the domain of your choice with hyves as the subdomain.

You will now get one of three different results for the pagerank of this non-existent subdomain -> PR7, PR4 and PR0.

PR7 means that the domain is perfectly fine
PR4 means that the domain got penalized by google
PR0 means that the domain got banned in Google

Sounds crazy right?  Well I was pretty skeptical until I tried it on a number of domains that I know that do and don’t have Google penalties. I know some have mentioned a few discrepancies, but all those I checked supported the conclusion above entirely. Weird.

To add a little more fuel to the fire, I thought I would run a mass pagerank check on subdomains of a number of directories. Now, we know Google started penalising low quality directories a while back, removed directories from their recommended guidelines and then put them back in again.

So before Google took away the ability to use the hyves trick I quickly scanned a load of directories on the old strongest directories list by Aviva. Old I know, but I wanted something to mass test. The results are as below.

Directories Not Penalised (Pr7)

Page Strength 10

http://www.dmoz.org

http://dir.yahoo.com

Page Strength 9.5

http://www.business.com

Page Strength 8.5

http://www.botw.org

http://bubl.ac.uk

Page Strength 8

Directory.v7n.com

Page Strength 7.5

http://www.ezilon.com

Page Strength 7

http://www.familyfriendlysites.com

http://www.goguides.com

Page Strength 6.5

http://www.bloggeries.com

http://www.dirjournal.com

http://www.elib.org

http://www.gimpsy.com

http://www.joeant.com

http://www.maxlinks.org

http://www.skaffe.com

http://www.tygo.com/dir

Page Strength 6

http://www.allinfodir.com

http://blogannounce.info

http://www.cascandra.com

http://www.dmegs.com

http://www.incrawler.com

http://www.kahuki.com

http://www.onlinewide.com

http://www.prolinkdirectory.com

http://www.rakcha.com

http://www.top5jamaica.com

http://www.ventedoy.com

http://www.webahead.net

Page Strength 5.5

http://www.anthonyparsons.com

http://www.blazemp.com/dir

http://www.ebusiness-directory.com

http://www.enquira.com

http://www.global-weblinks.com

http://www.linkwith.us

http://www.nzpages.co.nz

http://www.pegasusdirectory.com

http://www.seoma.net

http://www.web1directory.com

http://www.webotopia.org

http://www.zorg-directory.com

Page Strength 5

http://www.123kidzarea.com

http://www.alikedirectory.com

http://www.gbguide.com

http://www.homesalez.com/directory

http://www.linkspub.com

http://www.massivelinks.com/

http://www.postdotcom.com

http://www.qoobe.org

http://www.umdum.com

http://www.web10.ws

Directories Penalised (Pr4)

Page Strength 7

http://www.avivadirectory.com

http://www.romow.com

Page Strength 6.5

http://www.2yi.net

http://www.index-it.net

http://www.mygreencorner.com

http://www.relmaxtop.com

http://www.webworldindex.com

Page Strength 6

http://www.allydirectory.com

http://www.busybits.com

http://www.browse8.com

http://www.domaining.in

http://www.ebjuris.com

http://www.gii.in

http://www.sevenseek.com

http://www.site-sift.com

http://www.szab.net

http://www.uksmallbusinessdirectory.co.uk

http://www.wowdirectory.com

Page Strength 5.5

http://9ug.com

http://www.abilogic.com

http://www.apahcinc.org

http://ask-dir.com

http://www.authoritydirectory.com

http://blogaboutmysite.com/directory

http://blogtagstic.com

http://www.cannylink.com

http://www.dirwizard.com

http://www.emillie.net

http://www.e-topic.com/directory

http://www.idk.in

http://www.kwikgoblin.com

http://www.linkopedia.com

http://www.linksjuice.com

http://www.makeasearch.com

http://www.map100.com

http://www.nobledirectory.com

http://www.photarium.com

http://www.qango.com

http://www.vxbox.com

Page Strength 5

http://www.alivedirectory.com

http://www.businessseek.biz

http://www.ensuredirectory.com

http://www.ewebpages.org

http://www.excellentguide.com

http://www.fullofsearch.com

http://www.linkcentre.com

http://www.maxdirectory.eu

Directories Banned (Pr0)

http://www.uncoverthenet.com

So What Can We Takeaway From This Data?

First of all, it’s worth noting I haven’t had a chance to go through all the above and verify the results. Or that all the sites are working, what their current Pr is or whether they actually have a hyves subdomain etc. :)

But at a top level view, I think there is a clear difference in quality between the sets of directories which supports the hyves theory. There are a few low quality directories in the ‘not penalised’ group, but perhaps these simply got through the net.

If my memory serves me correctly uncoverthenet.com was banned from Google completely quite sometime ago and it still seems to be the case based on the above results. Their full domain is also a Pr0 and not showing for their own name.

Obviously don’t take this data at face value – I am not saying you should go and submit your site to any of the non penalised directories, far from it. Directory submissions are very low value these days, this was purely for analysis before Google took it away.

Key Points From Hyves -

• It’s pretty easy to spot a site that has been penalised already. So this data didn’t really tell me much more than you could already work out yourself from a quick analysis.
• There was not a lot of consistency on sites penalised – Some sites selling links but not others, some sites involved with text link ads but not others, some on the DP network, but not others etc. Again, like we already know!
• Another point to remember is that SOME of the pagerank penalisations were toolbar only at a superficial level and links from these sites may still have value.

Anything I missed?

As noted by Barry over at Seroundtable, apparently Google is now recommending that you do not have multiple match types for the same search phrase. So for example, you shouldn’t have the keyword ‘golf clubs’, set on broad, phrase and exact match types.

Broad -   golf clubs
Phrase – “golf clubs”
Exact -   [golf clubs]

Now I haven’t heard or read anything official from Google personally, but a member over at Webmaster World gives the Google representatives reasoning as -

“Instead of triples of all keywords, they want advertisers to go from “broad” to “narrow” (in their words).”

Although Google may have a point that every keyword does not need to be replicated for each match type it is suprising to hear (if true) because actually you should start completely the opposite way with your PPC campaigns. So ignore Google on this one and I will explain in more detail why. If you don’t already know what each match type does, read up here.

Start Narrow

Don’t waste your money by using the default broad match straight away. Start narrow with specific exact, phrase and negative keyword combinations before even thinking about using broad match.

You can start with all exact match phrase keywords before building out to phrase match, but if you are confident and use negative keywords properly phrase match is easy to use. Not all keywords will require an exact and phrase match equivalent either, but absolutely those keywords with medium to high volume should have an exact match version. Equally those lower volume keywords might not require a exact match version. So to take the golf clubs example, I might start with only an exact match version of [golf clubs] as it’s very high volume and could potentially appear against an even larger number of terms if on phrase. However, a lower volume term like “buy golf clubs uk” probably wouldn’t require an exact match version and you could start this on phrase match straight away.

This approach means you do not need a different match type for every keyword and helps to keep your campaigns managble when dealing with hundreds of thousands of keywords already. I know some agencies and bid management software companies recommend replicating keywords into each match type which is fine, but you will often find that your campaign is bloated and a large part of it is unnecessary.

The narrow to broad approach also applies to the actual keywords you are using aswell as the match type, so use specific keywords with long tail variations, rather than more generic general phrases to begin with.

Make sure your campaign is performing with these match types to start with – monitor your internal logs and run search phrase reports (to see what people really searched for) to expand your current keywords or add further negatives to your campaign.

Use Broad Match To Sweep Up

If you have additional budget or want more volume you can then think about introducing broad match, although the expanded match element does need to be very closely monitored.

Use broad match to ‘sweep up’ any low volume keyword variations that exact and phrase match types have not already covered. When using broad match, try initially going in with a lower bid than your exact and phrase match equivalents and it can produce great results. You will need to continue to monitor your logs and run search phrase reports to further expand your keywords (& negatives). Again, aswell as match type you can also start to trial more generic search phrases for further volume (on exact match of course, have you been listening?).

Splitting Adgroups Based On Match Type

Not always, but sometimes it makes sense to split up adgroups via match type. You know when using an exact match phrase that for the advert to appear that [exact phrase] must be searched for, so it makes sense to focus your advert on this term specifically. When using broad match, your advert might appear against a number of keyword variations so sometimes it makes sense to diversify your ads a little more and test. You might find that you just make a new adgroup for just one or a few of your exact match keywords as you need greater control over them for example. Remember, trial, review and test what produces the best CTR and more importantly conversion for your campaign.

Optimise Match Types Based On Performance

Obviously the great thing about using various match types is that each match type will perform differently from the other. So each should have their own separate bid and be optimised based on their own individual performance. This allows you to have greater control over your campaign and ultimately spend money where it makes the most sense for improved return.

Note – Don’t forget, the choice of match type does not impact your quality score. All match types for the same keyword have the same quality score regardless, so this is not a factor. They do obviously massively influence your CTR though, which is the biggest factor of Googles quality score algo.

Google have started testing Adwords ads within Google suggest, like this example below from Search Engine Land -

Normally Google do not charge for clicks on new advert testing like this which is good news. But for me it’s quite scary how far they are taking the expansion of where they show their adverts.

At the top of search suggest is far to easy for accidental clicks in my view although Google are also testing showing ads at the bottom which would seem better. Another concern for navigational brand searches, are Google offering you any real value in showing your advert here? Especially with increased use of Chrome which combines a traditional url bar with search.

Google recently removed keyword trademarking in the UK and subsequently profit through increased brand bidding from advertisers in fear of losing brand traffic to competitors.  In many cases advertisers do not need to bid against their brand or the money would be better spent elsewhere, so will these adverts fuel even more fear? Johanna Wright Director of Product Management at Google argues -

“This is another example of search becoming more dynamic and Google getting users to the correct results as fast as possible,”

Is Google really focusing on the user experience here or is this a revenue making exercise? Lets hope Google make this opt in or at least have an opt out feature at the time of its (potential) introduction, not months after.

It’s been a while since my last post. I have been really busy. But if you haven’t already heard Google have now updated their ad policy for hard alcohol and liquer targeting the U.S. I am told the UK will follow suit in the coming weeks.

Google had already quietly changed their policy at the end of October allowing PPC ads for beer. However according to the Inside Adwords blog the new rules are for promotion and branding rather than direct sale which are still not allowed -

“To comply with the updated hard alcohol and liqueurs policy, advertisers must promote the information about hard alcohol and liqueurs that their websites contain, such as recipes and brand messages. Ads that directly promote the sale of hard alcohol and liqueurs are still not permissible through our program. In contrast, advertisements for beer may directly promote its sale.“

The news comes after Google firstly removed their keyword trademark policy in the UK and then permitted advertising for online gambling in the UK, Italy, France and Spain.

Obviously only the cynics would correlate this with an attempt to subdue the effects of the current economic climate, Googles subsequent fall in share price and predicted fall in ad spend for 2009 where 97% of Googles revenue is still driven from. Although it is certainly good to see Google making decisions on a local level meaning big opportunities have opened up for those businesses (and in turn search marketing agencies) in these industries.

Yesterday evening there were some big changes in Google search results as discussed on Webmaster World and reported by Brian Turner and Aaron Wall. However, the algo update or data push has been rolled back and rankings are back to as they were, at least for now.

Google were obviously equally unhappy with the search results as I was last night. In my opinion they were as poor as they have been in the last few years. A lot of big, big sites homepages went missing from the search results completely and much for the worse. I put this down to a bad data push rather than an algo change, but its hard to say. It certainly wasn’t obvious link buyers or sellers (or network schemes) who were targeted in the industries I know, it was far more random and unpredictable. In some cases non secure (http) versions of homepages were replaced with secure (https) versions even when the secure version had zero internal/external links. Strange behaviour indeed.

I actually managed to catch a screen shot of the ‘PPC blog‘ SERPS early yesterday evening. Considering this site ranks no.1 in Google UK for this term (has done since its creation) with sitelinks, it was quite strange to see it drop all of a sudden to 5th place. Not only that, as you can see there were 3 results on the 1st page, 2 of which were indented and strangely one being the homepage.

I could theorise as to what Google were attempting to do if this was an update, but I really don’t need to. This site is as white hat as it comes, all editorial links with zero link purchases/selling or exchanges and just good old fashioned unique content. So if this site ranks badly in the SERP’s, Google is quite probably going in the wrong direction. In my opinion of course :)

More seriously, there were multi million pound businesses just dropping out of the search results completely in some verticals I monitor. Not just one or two, but a number of big players in each industry. Is this a sign of things to come? No, as the changes were rolled back. But there is something obviously brewing.

I think the majority of the time everyone would agree with a big fat no on allowing affiliates to bid against their brand. Why pay an affiliate a commission for something you will get 99% of the time anyway?

However, there are times where clients of mine have agreed to a select few affiliates to bid against their brand and it is something I have been asked about a number of times since.

If you are unsure of the reasoning behind the closed brand bidding group, the logic is that by allowing a small number of top performing affiliates to gain extra commissions from the merchants brand it allows these affiliates to bid broader on more generic keywords or pay a higher CPC to send increased traffic and sales the merchants way.

Due to the change in trademark policy on keywords in Google, this might in some cases make more sense in terms of supporting brand protection, especially in higher CPC markets with more competition. But separate from this issue, I find the affiliate brand bidding argument extremely weak on the whole. What irritates me more than anything about this concept is often the lack of real thought gone into it like most marketing or business decisions should have.

In using a closed group of brand bidders the merchants aim is ultimately to increase their traffic, conversions or revenue (or whatever else the goal might be) by effectively paying the affiliate additional commissions, which does make sense. That’s fine.

But like any marketing decision for this to be measured accurately in terms of return and performance the merchant needs to know -

• The increase in extra traffic, conversions, revenue (the end goal)
• The amount of extra cost/commission paid to the brand bidders
• Any other extra cost involved (time, tracking, management whatever)

So Why Is This So Often Flawed?

• Ignorance – Very often, none of the above are even properly considered let alone set up to be tracked and measured accurately to be able to determine the performance

• Trust In Tracking & Affiliates – Tracking and measuring increases from the data can be inherently difficult to gauge anyway due to the number of factors involved. But even more so with a closed group the merchant needs to be completely sure that tracking is accurate because quite feasibly an affiliate can set it up however they wish. How can you be sure that every branded advert is being tracked separately? You can’t.

• Brand Ownership – Merchants are allowing affiliates to be the face of their brand. They will need to dedicate some time to monitor and police the ads and make sure they are representing their brand accurately. Also, if there are many adverts against the said brand, this can equally confuse searchers.

• Lack of Control – For large scale brands there are huge variances in brand related traffic that cannot always be predicted or controlled – meaning on occasions affiliates could all of a sudden receive a huge amount of commissions direct from the brand without providing extra in return.

• Higher CPC – If the merchant is also running a branded PPC advert, what extra cost are will they now be paying due to the increased competition from their own affiliates?

So Why Do We Still See The Closed Group Brand Bidding Model?

There are numerous reasons from bad advice from affiliate managers to pressure to hit targets. However in some cases I have found that marketeers actually embrace the lack of transparency that affiliate brand bidding provides. That may sound strange, but there is a sweeping under the carpet mentality sometimes.

It means they can reach their targets without seeming to pay a higher cost to do it. It’s easier to hide marketing expense in what can be viewed as just improved affiliate sales by allowing affiliates to syphon from their own brand than actually just paying out a higher CPA/commission. There can be a blissful ignorance in gaining more sales using this method (at least more perceived sales) and far from enough thought or consideration which is scary.

Conclusion

A much more transparent method is to simply agree to pay a higher CPA or commission to the select top performing affiliates because this is effectively what the affiliate brand bidding concept is attempting to do. In a roundabout kind of way. However, this is much easier to budget, track and measure and offers the same basic premise of more money for more of whatever the said business goal is. Working closely with the affiliate and providing extra offers or discounts is also a great option.

Marketeers will still have the same overriding issue of measuring performance of what extra they are receiving against the extra outlay of cost – but it comes with much greater transparency and control. My advice would be, if you want to push affiliates and really get the most out of them and your campaign, don’t hide your true cost in a closed brand bidding group.

Do you have any arguments for a brand bidding group? I would be interested in hearing any. :)

Some interesting talk today of ways of using paid search to play with your competitors. I am sure many advertisers have noticed incorrect prices, promotions, erroring destination urls or just plain poor adverts for their own campaigns – whether it be down to an in-house team, their agency or affiliates.

It can be a very frustrating process tracking down who is running the offending advert and getting it amended to display accurate information.

The reason is anyone can run any advert as anyone else. I can run an advert with the display url set as any company in the world. So can anyone else.

Sounds strange that someone might be willing to pay to do this if they are an competitor. But by doing this a person or company can set about a negative ad campaign against their competitor. It could be a fairly minor thing to something much, much naughtier.

Take for example if I were a competitor of both DialAFlight and Flight Centre.

DialAFlight have just released a statement saying it is instituting legal proceedings against Flight Centre for infringement of their trademark.

Flight Centre say they have removed the ‘DialAFlight’ keyword phrase from their campaign. This is not the first time DialAFlight have warned Flight Centre apparently.

So imagine what a competitor can do to really wind this situation up by just displaying a little advert against DialAFlight as Flight Centre. I’d imagine that would not go down well.

Another example might be where all competition within a niche are playing nice, not bidding against each others brands out of respect. There is an almost unspoken mutual agreement in place of ‘don’t bid against me, I won’t bid against you’.

So you could shake it up a little and start running competitors adverts against your other competitors.

You might just find that the competitors you are running adverts against spot this and react by now advertising against the company you were acting as.

Time to remove your own advert and watch the company you have been advertising as now run their own advert against the competitor because they spot their advert.

Oh the fun.

I am certainly not recommending anyone to do this. In fact, I am recommending people not to do this as there are more intelligent ways to spend your advertising budget. But I have seen some cases of this over the past few months and it’s something to keep an eye as we might see more cases.

More on black hat PPC techniques here.