Hi,
Is there a single command to prevent a crawler from going into any subfolder? The reason I am asking this is, it seems the google is picking up subfolder that don't even exist on my site. So just excluding each folder is not enough , I need a general command.
Thanks

News