Implications of Disallowing A LOT of Pages
-
Hey everyone,
I just started working on a website and there are A LOT of pages that should not be crawled - probably in the thousands. Are there any SEO risks of disallowing them all at once, or should I go through systematically and take a few dozen down at a time?
-
Perfect, that's my intent. Thanks so much for your help!! I really appreciate it.
-
Correct.
Also, (i´ve should have said this earlier) there are no effective way to avoid robots crawling the web. The thing here (and what im answering) is to not show (and/or remove if are already in the index) certain pages in google search result.
-
That would happen if I used noindex, correct?
-
Exactly.
Also, remember that disalowing by robots.txt will not remove them from Google´s index.
-
LOL now I'm worried about what you mean by "correctly." Do you mean just making sure that I'm not disallowing valuable pages that should be crawled?
-
Hello Rachel,
If you do it correctly, this is not disallowing useful pages, there is no risk. Go ahead.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site Crawl -> Duplicate Page Content -> Same pages showing up with duplicates that are not
These, for example: | https://im.tapclicks.com/signup.php/?utm_campaign=july15&utm_medium=organic&utm_source=blog | 1 | 2 | 29 | 2 | 200 |
Technical SEO | | writezach
| https://im.tapclicks.com/signup.php?_ga=1.145821812.1573134750.1440742418 | 1 | 1 | 25 | 2 | 200 |
| https://im.tapclicks.com/signup.php?utm_source=tapclicks&utm_medium=blog&utm_campaign=brightpod-article | 1 | 119 | 40 | 4 | 200 |
| https://im.tapclicks.com/signup.php?utm_source=tapclicks&utm_medium=marketplace&utm_campaign=homepage | 1 | 119 | 40 | 4 | 200 |
| https://im.tapclicks.com/signup.php?utm_source=blog&utm_campaign=first-3-must-watch-videos | 1 | 119 | 40 | 4 | 200 |
| https://im.tapclicks.com/signup.php?_ga=1.159789566.2132270851.1418408142 | 1 | 5 | 31 | 2 | 200 |
| https://im.tapclicks.com/signup.php/?utm_source=vocus&utm_medium=PR&utm_campaign=52release | Any suggestions/directions for fixing or should I just disregard this "High Priority" moz issue? Thank you!0 -
One page templates
Hi, I have in plan to implement One-page template (http://shapebootstrap.net/wp-content/plugins/shapebootstrap/demo.php?id=380) is someone have experience how this type of page have SEO problems? Best regards
Technical SEO | | komir20040 -
Pages not being indexed
Hi Moz community! We have a client for whom some of their pages are not ranking at all, although they do seem to be indexed by Google. They are in the real estate sector and this is an example of one: http://www.myhome.ie/residential/brochure/102-iveagh-gardens-crumlin-dublin-12/2289087 In the example above if you search for "102 iveagh gardens crumlin" on Google then they do not rank for that exact URL above - it's a similar one. And this page has been live for quite some time. Anyone got any thoughts on what might be at play here? Kind regards. Gavin
Technical SEO | | IrishTimes0 -
Page feedback
We recently wrote a new website page to cover the direct mail services our organization offers. We kept the title tag to 70 characters, the meta description under 150 characters. H1 tag has what we feel is the most important term. If anyone out there has time to review & provide a little feedback, we'd really appreciate it. It would be great to know if it is built well and providing a solid end user experience. http://www.cushingco.com/print_products/additional_services/direct_mail.shtml At the moment, the only links pointing to this page are from our blog. One bit of content I am contemplating is a short paragraph - What is Direct Mail Marketing? Literally providing a short definition of it. The page was activated last Thursday and showing up in some Google results on the 4th/5th page but I am thinking this is probably just a temporary bump for now. Anyway, thanks in advance for any advice!!!
Technical SEO | | SEOSponge0 -
Home Page .index.htm and .com Duplicate Page Content/Title
I have been whittling away at the duplicate content on my clients' sites, thanks to SEOmoz's pro report, and have been getting push back from the account manager at register.com (the site was built here and the owner doesn't want to move it). He says these are the exact same page and he can't access one to redirect to the other. Any suggestions? The SEOmoz report says there is duplicate content on both these urls: Durango Mountain Biking | Durango Mountain Resort - Cascade Village http://www.cascadevillagehotel.com/index.htm Durango Mountain Biking | Durango Mountain Resort - Cascade Village http://www.cascadevillagehotel.com/ Your help is greatly appreciated! Sheryl
Technical SEO | | TOMMarketingLtd.0 -
Search Result Page, Index or Not?
I believe Google doesn't want to index and show other search result pages in there SERP.
Technical SEO | | DigitalJungle
So instead of adding "noindex, follow" tag i have changed the url in my search result page like this: Original
http://www.mysite.com/kb-search.aspx?=travelguide&type=wiki&s=3 To
http://www.mysite.com/travelguide/attraction-guide.html And the search result page contains the title of the articles, a short descriptions (300 chars.) and a link to the articles. Does it help? Or should i add noindex, follow tag? Helps Please?0 -
Are there any SEO implications if a page does two 301s and then a 304?
Curious to see if this is a positive or negative thing for SEO...or even perhaps, neutral. h9SZz
Technical SEO | | RodrigoStockebrand0 -
How to implement a temporary splash page?
Hello - Our company will soon be launching a major product enhancement and we have discussed using a splash page when a visitor hits our homepage to promote the launch. I'm thinking something similar to what Apple did on Apple.com to announce addition of The Beatles catalog ot iTunes. I've never actually implemented anything like this before and wanted to get feedback from the community on how best to handle this from an SEO perspective. Thanks, Jason
Technical SEO | | TKSearchGuy0