[Fsf-friends] LXer: Is the MSN search engine being used in the battle to killLinux? (fwd)

[Frederick Noronha (FN)]

---------- Forwarded message ----------

http://lxer.com/module/newswire/view/23328/index.html

Hello,

I could not track the source of old content showing up at the top of google searches on my name. According to the number of hits I generate, I have a significant mind share on Google. Finally, we tracked down the problem and found that MSNbot was hammering and multiplying links old and irrelevant content, pushing that above recent articles about Linux in Government, case studies, etc.

I also found other open source advocates on the web trying to find out why they couldn't stop MSNbot using a standard robots.txt directive in the root directory of their web sites. Some of the complaints related to increased fees for bandwidth.

In this article, I discuss the issue. I will also post the fix we discovered by accident. BTW, a standard redirect will not work.

I'm not very pleased to discover Microsoft is manipulating the content on the Internet.

Tom Adelsten