Pages of each website regularly manage search robots who process information which is on the website that it was available on requests of users in search engines. Visits of the visitors who came from searchers positively affect attendance of the website and stimulate development of a resource. But, if on the website there are pages on which information is not intended for Internet audience (for example, personal pages of users or the login page), then transition from searchers to such pages is not necessary and it is better to prohibit this information to indexation.
It is required to you
- - have own website
- - know links to pages which indexation needs to be prohibited.
- - have at least initial knowledge of HTML
Instruction
1. To prohibit the search robot to index certain pages website, it is necessary to make changes in the robots.txt file if it already is on your website, or to create this file.
2. If the robots.txt file is already uploaded on the website and it is possible to change this file directly from the control panel of a resource, then open for robots.txt through web service for editing.
3. If there is no opportunity to make changes to the necessary file directly from the control panel, then open for robots.txt in the browser, having followed the link to this file. Copy all data from the opened page and insert on a leaf in the Notepad program.
4. In the absence of robots.txt on the website, open the Notepad program in which new document it will be necessary to enter the necessary code.
5. On the empty sheet of the text document write the first obligatory line:
User-agent: *
6. Then pass to the following line, having clicked the ENTER key, and type the following code:
Disallow: / index/page3Disallow: designates the indexation ban, and / index/page3 is a link to the page which does not need to be indexed. Instead of link/index/page3 insert the link to the page of your website which needs to be prohibited for processing by robots.
7. Notice that at the beginning of the link there is no name of the domain, the oblique line means its existence in record. The search robot at a website round automatically substitutes a domain name in such links. Therefore write links to the necessary pages thus.
8. After Disallow: it is possible to place not only the link to the specific page, but also the link to any section or a directory of the website. In this way it is possible to prohibit indexation of the necessary part of the website.
9. In need of the processing ban by robots of several different sections of a resource, write the link to each section or the page from a new line at the beginning of which do not forget to place the record Disallow:.
10. To complete change of robots.txt, click "Save" when editing the file in the control panel or save the text document with records under the name of robots.txt and upload this file on the website.