Blocking BAIDUSPIDER | Develop Site

fumadrid's picture

Baiduspider bot is a Chinese search engine Baidu. This generated a lot of traffic and taking up too much bandwidth. We have tried to block it using robots.txt file but only worked for a while.

According to Baidu specified on your page, with just an update of the robots.txt file would suffice.
adding:

User-agent: Baiduspider
Disallow: /

But so far yielded no results. In any case clear that this can take several months

After a while we started to see results and traffic from China decreased significantly by changes in the file ROBOTS.TXT and .HTACCESS

English