Rupert vs. Google – Interesting

I listened with interest to David Speers (@David_Speers) talking to Rupert Murdoch on Sky News Australia (@SkyNewsAust) yesterday… a story which has been picked up globally now… Listen for yourself here:  http://twaud.io/r35

The most interesting part, and the part picked up across the web was Mr Murdoch’s statement that he would consider and be looking at removing his sites from the Google Index.

What does that mean?

Ok – So, anyone who owns a website can CHOOSE not to be displayed on Google – just the opposite to what most people try for – and as such nothing on your site will show up anywhere in Google – Search, Images, News.

It’s called Robots.txt – a small file you place on your server which ‘tells’ the robots (the computers which constantly trawl the internet for content to simply NOT search your site.

In fact, Google’s very up front about this – http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=93710

You can read there how to do it.

But why would you!

Well, one good reason is because you are trying to monetise the content on your website, and by allowing google to ‘index’ your site, you are allowing free access to all or part of your content.

Rupert’s pay wall argument was always going to come up against some stiff opposition – but now, I wonder if he might be onto something.

I watched with interest a clip from Jason Calacanis’ web show This Week in Startups – which has in it a discussion about how the big content providers could ‘gang up’ and cause a shift from one search engine to another (google to Bing for example) through a simple business deal.  That clip is below.  Question is – was Rupert onto this from the start?  Is he angling for such a deal, or simply a deal just like Twitter has recently done with Bing and Google, allowing them access to the ‘twitter content’ or ‘tweet stream’ for search indexing.

Very interesting times indeed.