robots.txt SEO for your WordPress Blog

Did you know that through robots.txt you can control which post or page of your site to crawl or not to crawl? and if you’re currently worried about duplicate content on your site, well not a problem folks, robots.txt is also helpful to avoid duplicate content, yes it is possible to avoid duplicate content via robots.txt by not crawling site directories, like tag, archive and category these can cause duplicate content.

Implementing good SEO robots.txt will help you get high traffic in most search engines or even receive more paying in relevant ads.


Copy and paste the below snippet into robots.txt of your site.

header.php trick

Add this snippet in header.php of your current theme, we simply add conditional statement which robots to display in our blog.

Don’t want to get your hand dirty?

Don’t worry folks as always I will provide plugin from other developer, if there’s any 🙂
KB Robots.txt

by Ryan Sutana

  • Ted

    Nice post, thanks for sharing =)

  • I used to be very happy to find this internet-site.I wished to thanks to your time for this wonderful read!! I undoubtedly enjoying every little bit of it and I have you bookmarked to check out new stuff you weblog post.

  • I really enjoyed reading your blog. You have good views, Keep up the good informative info. Thanks a lot for sharing this post.

  • There are certainly a lot of details like that to take into consideration. is really good. That is a great point to bring up. I offer the thoughts above as general inspiration but clearly there are questions like the one you bring up where the most important thing will be working in honest good faith. I don?t know if best practices have emerged around things like that, but I am sure that your job is clearly identified as a fair game. Both boys and girls feel the impact of just a moment’s pleasure, for the rest of their lives.

  • It is perfect time to make some plans for the future and it’s time to be happy. I’ve read this post and if I could I want to suggest you few interesting things or tips. Maybe you could write next articles referring to this article. I want to read even more things about it!

  • Nice information, valuable and excellent design, share good stuff with good ideas and concepts, lots of great information and inspiration, both of which we all need. Thanks for your fantastic blog.