Ecommerce site, around 18k products. Logs show Googlebot hitting /search?q= URLs about 50k times a day. Some of those query strings are garbage, some are clearly spam from bots that submitted weird searches and got the pages linked somewhere.
About 3,000 of them are indexed according to a site: check, and a few actually get impressions for long tail stuff.
My plan:
– robots.txt disallow for /search
– stop rendering search links in the “popular searches” block
– turn the 30 or so search pages that get impressions into real category or landing pages
The part I’m unsure about is the order. If I disallow first, the indexed ones can’t be recrawled to see a noindex, so they may hang around. If I noindex first, it keeps eating crawl for weeks.
What order did you do it in, and how long did it take for the indexed ones to clear?
.
