Should I include unnecessary pages in the sitemap.xml
-
I have a lot of pages that I don't want Google to index, so for most of them, I used cannonical, were they were duplicates, noindex were I wanted to remove the pages, but the question is: Should I include these pages in the sitemap.xml, or just the important pages?
Also should I include them in order to get the changes indexed fastet by Google?
-
That clearly changes my ideas about this ;-). As we're talking about a couple of million pages I wouldn't include them in the sitemaps then and to make sure they're absolutely made sure that it's noindexed.
-
One of the main problem is that there are a lot of such pages (aprox. 2-3 milions) and my indexation rate is really slow for a site this big. The old sitemap structure was to complex, and I wanted so simplify it, so Google wiil crawl only the important pages
-
Hi Silviu,
Hard question, related to your use case I would suggest not to include them. But on the other hand it also shouldn't harm your performance as the URLs in a sitemap are mostly meant to search engines as a full list of URLs they might miss otherwise. It would also help you to see what your indexation rate is. Curious to see what other people think about this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Any recommendations for an XML Sitemap for a large community website?
Hi all, Once of our clients is a large community website for parents/parenting. The standard Wordpress XML Sitemap plugin is throwing up lots of errors, etc, and is not ideal. Does anyone have any recommendations for either a tool that we could use to create a better one, or else a service that we could pay to use? Gavin
On-Page Optimization | | IcanAgency0 -
Need Suggestion for Canonical Page
Hello, I am bit confused about whether to use a Canonical URL on a page or not? Actually, the project I am working on is having two pages with most similar content. The only difference between them is that only 1 paragraph of 50-60 words is different. I am not sure, whether to put a canonical URL on the another version of the page. [Note: Sorry, can't put the site URL due to some restrictions.]
On-Page Optimization | | Anup_More0 -
Duplicate Page Content
Hi there, We keep getting duplicate page content issues. However, its not actually the same page.
On-Page Optimization | | HamiltonIsland
E.G - There might be 5 pages in say a Media Release section of the website. And each URL says page 1, 2 etc etc. However, its still coming up as duplicate. How can this be fixed so Moz knows its actually different content?0 -
"Issue: Duplicate Page Content " in Crawl Diagnostics - but these pages are noindex
Saw an issue back in 2011 about this and I'm experiencing the same issue. http://moz.com/community/q/issue-duplicate-page-content-in-crawl-diagnostics-but-these-pages-are-noindex We have pages that are meta-tagged as no-everything for bots but are being reported as duplicate. Any suggestions on how to exclude them from the Moz bot?
On-Page Optimization | | Deb_VHB0 -
How to treat pages that are removed?
I have a website that need be very up-to-date, I mean, pages can be published just for 30 days, after that it should be unpublished. Everyday more than 300 pages is "removed", For theses pages I am returning http code "410" (Gone), also I remove from the sitemap. Now, I am checking Google WebMasterTools and I am getting thousands of pages not found. So... My questions Does it have SEO impact? How is the best approach to treat it?
On-Page Optimization | | thobryan0 -
Why isn't our site being shown on the first page of Google for a query using the exact domain, when its pages are indeed indexed by Google
When I type our domain.com as a query into Google, I only see one of our pages on the homepage, and it's in 4th position. It seems though, that all pages of the site are indexed by google when I type in the query "site:domain.com". There was an issue at the site launch, where the robots.txt file was left active for around two weeks. Would this have been responsible for the fact that another domain ranks #1 when we type in our own domain? It has been around a couple of months now since the site was launched. Thanks in advance.
On-Page Optimization | | featherseo0 -
Changing the url of a page
Hello. I would like to change the url of a page. It currently has very few inbound links. I would set up a 301 redirect to the new url. Is there anything else I should take into account before changing the url? Is there a downside to changing a url? Do inbound links carry the same value when a 301 redirect is involved? Thank you!
On-Page Optimization | | nyc-seo0 -
Duplicate Page Title
I have a dating site, it's got a lot of duplicate page titles, most of them are the language buttons for the users to view the site in there language. but I think it's obvious that the buttons don't have anything to do with it. I'm thinking that page tittle is basically a description of what the site is. like for an example "online-dating" is this it? please tell me in terms for a dummy, how to fix it.
On-Page Optimization | | clickit2getwithit0