by strayinma » Sun Nov 16, 2008 9:25 am
Googlebot will continually revisit your site and crawl linked pages. You can help/improve this process by implementing the Google Sitemap, an XML file managed in your Webmaster toolset. Totally legal - it helps manage the process. It does not help your ranking, just getting crawled.
Learn about the sitemap (and protocol) here:
https://www.google.com/webmasters/tools/docs/en/protocol.html
If you have a free Google Webmaster tools account, you can see when GGbot last visited your site and see where it tripped over your site while crawling (and what the roadblock is so that you can fix any site errors).
You can also see what has/has not been indexed.