DE BASISPRINCIPES VAN ORGANISCH VERKEER

To avoid undesirable inhoud in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory ofwel the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search eng