From 4bbd7f531017daf2b72517b1c1e20b523270bdf8 Mon Sep 17 00:00:00 2001 From: Grant Slater Date: Wed, 3 Mar 2010 11:41:43 +0000 Subject: [PATCH] Update robots.txt to Disallow browser [node|way|relation] history. Content now worth indexing in most cases. --- public/robots.txt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/public/robots.txt b/public/robots.txt index 8d2bea7ea..496aa201e 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -29,6 +29,9 @@ Disallow: /traces/tag/ Disallow: /traces/page/ Disallow: /api/ Disallow: /edit +Disallow: /browse/node/*/history +Disallow: /browse/way/*/history +Disallow: /browse/relation/*/history Allow: /browse/node/ Allow: /browse/way/ Allow: /browse/relation/ @@ -52,6 +55,9 @@ Disallow: /traces/tag/ Disallow: /traces/page/ Disallow: /api/ Disallow: /edit +Disallow: /browse/node/*/history +Disallow: /browse/way/*/history +Disallow: /browse/relation/*/history Allow: /browse/node/ Allow: /browse/way/ Allow: /browse/relation/ -- 2.39.5