Should I include unnecessary pages in the sitemap.xml
-
I have a lot of pages that I don't want Google to index, so for most of them, I used cannonical, were they were duplicates, noindex were I wanted to remove the pages, but the question is: Should I include these pages in the sitemap.xml, or just the important pages?
Also should I include them in order to get the changes indexed fastet by Google?
-
That clearly changes my ideas about this ;-). As we're talking about a couple of million pages I wouldn't include them in the sitemaps then and to make sure they're absolutely made sure that it's noindexed.
-
One of the main problem is that there are a lot of such pages (aprox. 2-3 milions) and my indexation rate is really slow for a site this big. The old sitemap structure was to complex, and I wanted so simplify it, so Google wiil crawl only the important pages
-
Hi Silviu,
Hard question, related to your use case I would suggest not to include them. But on the other hand it also shouldn't harm your performance as the URLs in a sitemap are mostly meant to search engines as a full list of URLs they might miss otherwise. It would also help you to see what your indexation rate is. Curious to see what other people think about this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Contact pages coming up for keywords above landing pages
I have two examples of contact pages coming up over designated landing pages Keyword: Nickel Alloys for www.neonickel.com Keyword: Artificial Grass for www.artificialgrass4u.co.uk Is there anyway I can stop this happening?
On-Page Optimization | | icansee0 -
Too Many On-Page Links
Hello. So, my SEO team has worked very hard to finally resolve RogerBot/GoogleBot specific Crawl Errors either manually or programmatically can be fixed for our Budget Blinds USA Pro Campaign. We've done a good job even if a lot of it came from Robots.txt file entries as this was the most efficient way our client chose to do it. Good news is most of it is CMS configuration and not bad site architecture. That being said our next big volume of Crawl Errors is "Too Many On-Page Links". Our Moz DomainRank is 61. Our client, on this new version of the website, added a large nav-based footer which has duplicate links from the Header Main Navigation. I believe our solution is to put in No-Follow Metatags at the Footer Link Level, so we don't zap Page Authority by over-dividing as you recommend. Is this the best way to resolve this? Is there any risk in this? Or is a 61 DomainRank high enough for RogerBot and GoogleBot to crawl these anyway? Please advise,
On-Page Optimization | | Aviatech0 -
Duplicate Page Title
I have a dating site, it's got a lot of duplicate page titles, most of them are the language buttons for the users to view the site in there language. but I think it's obvious that the buttons don't have anything to do with it. I'm thinking that page tittle is basically a description of what the site is. like for an example "online-dating" is this it? please tell me in terms for a dummy, how to fix it.
On-Page Optimization | | clickit2getwithit0 -
Can we listed URL on Website sitemap page which are blocked by Robots.txt
Hi, I need your help here. I have a website, and few pages are created for country specific. (www.example.com/uk). I have blocked many country specific pages from Robots.txt file. It is advisable to listed those urls (blocked by robots.txt) on my website sitemap. (html sitemap page) I really appreciate your help. Thanks, Nilay
On-Page Optimization | | Internet-Marketing-Profs0 -
Pages vs Posts
What are your thoughts on pages vs posts? I am setting up a new blog for a client but not sure how to structure the content. I may just do posts or a whole bunch of page listed down the sidebar. It seems like my pages always rank better than my posts. Has anyone else noticed this? Could it be because of the dates tied posts?
On-Page Optimization | | SixTwoInteractive0 -
Different pages for OS's vs 1 Page with Dynamic Content (user agent), what's the right approach?
We are creating a new homepage and the product are at different stages of development for different OS's. The value prop/messaging/some target keywords will be different for the various OS's for that reason. Question is, for SEO reasons, is it better to separate them into different pages or use 1 page and flip different content in based on the user agent?
On-Page Optimization | | JoeLin0 -
Why does my on-page report card say my page title is 403 forbidden when its not?
I'm trying to get on top of my on page stuff and I'm going through the SEO Moz on-page report cards and it says I'm scoring a fail on certain elements within the 'critical' and 'high importance' factors as my page title is '403 forbidden' but when I go on to my site, my sites CMS it's not '403 forbidden' it's the text I entered?
On-Page Optimization | | jamesj35mm0 -
Optimization of home page
Hi there I have an issue which, despite searching hard, I simply cannot find the right solution for. We have an index page that used to rank pretty well for a main industry keyword. However following a revamp of the site last year the kw slipped and no longer brings in decent traffic levels. The problem seems to be that the old static site had a sprinkling of variable anchor text links that brought value to the home page. Instead of the main anchor being "home" we would revert to "main keyword" and variations across the site sometimes in t he content but mainly on the nav bars. However the new CMS design structure restricts us considerably with anchor distribution and so instead we opted for the site logo on the masthead to have an ALT tag for "main keyword" but so as not to game google too much we added .."home" to the tag. Probably pointless but we figured it could do no harm. This ALT text is site wide Problem now is that we have lost the spread of internal nav bar anchors and variety etc. We have slipped in the serps for "main keyword" and I cant help thinking we are not maximising the anchors as we should. So what Im coming to is this.... How can we tell if Google is picking up the ALT tage anchor as the main anchor to rank the site at the expense of all internal text anchors. Despite retaining lots of embedded anchors - according to the Moz metrics these are not being picked up because OSE suggests the ALT tag anchor is taking precedence. The serps probably support this view as well. Should we: a) Vary the masthead ALT if there is no way of avoiding this being the most important link / anchor on the page b) Remove the ALT anchor and instead opt for content links high on the page (we do have nav bar links saying "Home" site wide as well which may overrid the embedded links?) c) Leave the ALT alone and still push for content anchors as described in b) What is the best way to handle this..? Best wishes and thanks Morch
On-Page Optimization | | Morch0