Some of this depends on crawl rate and the number of pages you are aiming to de-index. We ran an exercise in which we aimed to de-index 40k pages from our site. It took two months before we saw at least 50% of the pages de-indexed, while it was 6 months before we even saw 75% of the pages de-indexed. However, these were pages that were part of a subdomain with very little authority and low crawl rates. We performed the same exercise with a smaller set of pages, but much higher crawl rates and higher domain authority, and saw an almost instantaneous (24-48 hour) impact in which they were removed. As the mobile site sounds fairly new, you may just have to be patient on it as it will likely take longer to de-index.
- Home
- IHSwebsite
Latest posts made by IHSwebsite
-
RE: How long does it take for Google for deindexing pages?
-
Robots.txt: Can you put a /* wildcard in the middle of a URL?
We have noticed that Google is indexing the language/country directory versions of directories we have disallowed in our robots.txt.
For example:
Disallow: /images/ is blocked just fine
However, once you add our /en/uk/ directory in front of it, there are dozens of pages indexed.
The question is: Can I put a wildcard in the middle of the string, ex. /en/*/images/, or do I need to list out every single country for every language in the robots file. Anyone know of any workarounds?
Best posts made by IHSwebsite
-
RE: How long does it take for Google for deindexing pages?
Some of this depends on crawl rate and the number of pages you are aiming to de-index. We ran an exercise in which we aimed to de-index 40k pages from our site. It took two months before we saw at least 50% of the pages de-indexed, while it was 6 months before we even saw 75% of the pages de-indexed. However, these were pages that were part of a subdomain with very little authority and low crawl rates. We performed the same exercise with a smaller set of pages, but much higher crawl rates and higher domain authority, and saw an almost instantaneous (24-48 hour) impact in which they were removed. As the mobile site sounds fairly new, you may just have to be patient on it as it will likely take longer to de-index.
Looks like your connection to Moz was lost, please wait while we try to reconnect.