htaccess is an Apache Web Server configuration file. htaccess is located at the root of your web directory. Today we are going to use htaccess to ban bad bots from crawling and fetching our website data. Banning bad bots is not only useful to protect your content but also to reduce server load and stop utilization of our resource to serve bad bots. We can also ban by robots.txt but most of the bad bots ignore robots.txt and intentionally try to fetch prohibited content by robots.txt , so robots.txt is not useful in this case.
Generally, local business sites are targeting to maximum one or two countries, in this case banning rest of countries can save lots of bandwidth utilization. But this is not recommended as this may prevent users from other countries accessing their website and also some bots as Facebook may also not able to fetch website content.
We have seen lots of crawling bots and fake unusual traffic from China on tons of websites, this can be prevented by modifying the htaccess file. ip2location provides the latest lists of IP addresses which can be placed in htaccess to ban China traffic.
deny from 18.104.22.168/23
deny from 22.214.171.124/32
deny from 126.96.36.199/32
allow from all
It is an example of code inside htaccess to ban specific IP and give access to rest of IP. You can generate your customize htaccess to prevent traffic and bots from any countries and upload on your server.
However, this method is very useful if your traffic is over utilizing your resources and due to bad bots your real customers are facing issue while surfing website. If your site has unlimited bandwidth but your resources are very limited like less RAM or CPU, though I suggest you to ban bad bot so your real-time visitors do not feel lagging while using your website. At MR Cloud Hosting we provide 1 GB RAM, 1 CPU, and Unlimited bandwidth to keep your site smooth performance.
To edit the .htaccess file, first we will open the File Manager:
Warning, modifying htaccess wrongly may down your site. Must take the backup before making any modification.
If you are not able to find htaccess in public you can create new. If you face issue while creating htaccess, it means the file already exists but it is hidden. To show hidden files you have to click on setting option given at right top corner of CPanel File Manager.
Login to cPanel.
Click on “File Manager” in the “Files” section.
Select “Document Root for: (public_html)” and select the domain name you wish to access.
Make sure “Show Hidden Files (dotfiles)” is checked.
Click the “Go” button.
The File Manager will open in a new tab or window.
Look for the .htaccess file in the list of files. You may need to scroll to find this file.
Right click on the file and click on “Code Edit” on the menu.
A dialogue box may appear asking you about encoding. Just click on the “Edit” button to continue.
Edit the file as you require.
Click on the “Save Changes” button in the upper right-hand corner when done.
Test your website to make sure it has the desired effect. If not, correct the error or revert back to the previous version until your site works again.
Once complete, you can click “Close” to close the window.
Let us know if you have any suggestion or feedback by commenting below.
20 Nov, 2018