Thursday, 13 March 2014
If you need to cache static content in one of your sub folders use the following snippet of code placed into an htaccess file.
posted on Thursday, 13 March 2014 18:22:13 (South Africa Standard Time, UTC+02:00)  #    Comments [1] Trackback
Now for those robots which are bad and do not adhere to your robots.txt's rules, you'll need to add specific htaccess code.
posted on Thursday, 13 March 2014 18:18:02 (South Africa Standard Time, UTC+02:00)  #    Comments [0] Trackback
If you need to block all bots except for those that matter, look at this modified robots.txt snippet.
posted on Thursday, 13 March 2014 18:12:35 (South Africa Standard Time, UTC+02:00)  #    Comments [0] Trackback