|
Software_to_help_you_create_a_robotstxt_file
| Software to help you create a robots.txt file
Robogen is a software for Windows 95, Windows 98 and Windows NT
based computers which can help you create robots.txt files
without having to worry about the correct syntax of a robots.txt
file. It comes in two editions: the free Limited edition and the
Standard edition which costs $12.95. You can get Robogen from
http://www.rietta.com/robogen/.
In this article, I'll tell how you can use Robogen to simplify
the process of creating a robots.txt file. This article assumes
that you have read my article on what a robots.txt file is and
how you can manually create it. If you missed the article, it is
available at
http://www.1stSearchRanking.com/t.cgi?4901&robots.htm
I would recommend that you download the trial version of the
Standard Edition. The Limited edition does not support as many
search engines as the Standard Edition.
Once you have downloaded Robogen and installed it in your hard
drive, start the program. Here are the steps that you should
follow to create a robots.txt file:
Step 1: Click on the Logon FTP Server button. In the dialog box
that appears, type in the address of the FTP server for your web
site, make sure that the "This server requires me to log in"
option is checked, and then type in the username and the
password which you use to access the server in the appropriate
boxes.
Now, depending on how your web host has set up the server, the
files for your web site may be present in the root directory of
the server itself. Or, it may be present in a sub-directory of
the root directory. In the latter case, click on the "Use
specified root" option and type the relevant sub-directory in
the text box besides the option. If you don't know in which
sub-directory the files for your web site are present, ask your
web host to give it to you.
Done? Now click on OK. After a while, Robogen will display the
directories and files which are present in your site in the
right pane of the main Robogen program window. If you find that
Robogen has not displayed the files in your site, it means that
you did not enter the correct sub-directory in which the files
of your web site are present. In this case, ask you web host to
give the correct sub-directory to you.
Step 2: In the left pane of the main Robogen window, click on
the small button present to the right of the drop-down combo box
at the top. In the menu that appears, select "Popular Search
Engines".
Step 3: Now, click on the drop-down combo box and choose the
search engine that you are targeting (say, AltaVista). In the
right pane, select a page which is NOT meant for AltaVista. Now
click on the "
|
|
| |
| |
