How to block bad bots from accessing the website using .htaccess

Question

In the log file of my website, I see many bad bots and crawlers accessing my website. I have disallowed some of them in the robots.txt, but it seems that they are not obeying robots.txt. How can I block them using the .htaccess file?

Answer 1

You can use the HTTP_USER_AGENT to check those bots or spiders and block them. Here I am blocking some of the well-known bad bots. When these bots will try to access the page, 403 will be returned.

## block these bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (AhrefsBot|SemrushBot|Barkrowler|MJ12bot|DotBot|Java|DataForSeoBot|BLEXBot) [NC]
RewriteRule .* - [F,L]