Blocking BAIDUSPIDER | Develop Site
Baiduspider bot is a Chinese search engine Baidu. This generated a lot of traffic and taking up too much bandwidth. We have tried to block it using robots.txt file but only worked for a while.
According to Baidu specified on your page, with just an update of the robots.txt file would suffice.
adding:
User-agent: Baiduspider
Disallow: /
But so far yielded no results. In any case clear that this can take several months
After a while we started to see results and traffic from China decreased significantly by changes in the file ROBOTS.TXT and .HTACCESS
English
- fumadrid's blog
- Log in or register to post comments
- Español