05-18-2006, 06:30 PM
I'm sure you have, but just in case you could check the robots.txt is correct?
Its been a long time since i wrote a robots.txt but should it be -
Multiple forms of disallow (forum/, /forum/, /forum, forum ) to make sure robots really ignore it.
Code:
User-Agent: Googlebot-Image
Disallow: /
Its been a long time since i wrote a robots.txt but should it be -
Code:
User-Agent: *
Disallow: /index.php
Disallow: /forum/
Disallow: forum/
etc...
Multiple forms of disallow (forum/, /forum/, /forum, forum ) to make sure robots really ignore it.