Using robots.txt file.

andy - 06 Apr, 2013 3562 Views 0 Comment

Sometimes, when you develop your client website and publish the site as staging site. You want to make sure the site is not going to be crawled or indexed by Google or search engine. Otherwise it is going to looks bad, the main reason is because the site is not live and approved yet. So all the contents should be privately viewed by clients and yourself only.

In this tutorial, you will learn how you can create your own robots file and place it on the root of your website.

To block the site is pretty simple, what you need to do is to create a robots.txt and type in the following text:

User-agent: *
Disallow: /

Alternatively you can place this meta tag information on your header meta tags as well

<meta name="robots" content="noindex"/>

If you want to target Google search engine only, you can use the following meta tags.

<meta name="googlebot" content="noindex"/>


There are no comments available.

Write Comment
0 characters entered. Maximum characters allowed are 1000 characters.

Related Blogs

Related Tutorials

Site Navigation

Site navigation can be considered as another important part of search engine.

SEO Basic

Learn what are the basic of SEO.