Un robots.txt interesant
Dar poate ştiaţi de el… Iată cîteva spicuiri: User-agent: * Disallow: /cgi-bin Disallow: /search Disallow: /query.html Disallow: /help Disallow: /360pics/text Disallow: /911/911day/text Disallow: /911/heroes/text Disallow: /911/messages/text Disallow: /911/patriotism/text Disallow: /911/patriotism2/text Disallow: /911/progress/text Disallow: /911/remembrance/text Disallow: /911/response/text Disallow: /911/sept112002/text Disallow: /911/text Disallow: /ConferenceAmericas/text Disallow: /GOVERNMENT/text