If you need to block all bots except for those that matter, look at this modified robots.txt snippet.
User-agent: Googlebot
User-agent: msnbot
User-agent: bingbot
User-agent: AdsBot-Google
Disallow:
User-agent: *
Disallow: /
This script allows Google and Microsoft to index your website and will
help to radically reduce bandwidth consumption on your MyBB forum.