Submitting sitemaps every 7 days
-
Question, if you had a site with more than 10 million pages (that you wanted indexed) and you considered each page to be equal in value how would you submit sitemaps to Google?
Would you submit them all at once: 200 sitemaps 50K each in a sitemap index?
Or
Would you submit them slowly? For example, would it be a good idea to submit 300,000 at a time (in 6 sitemaps 50k each). Leave those those 6 sitemaps available for Google to crawl for 7 days then delete them and add 6 more with 300,000 new links? Then repeat this process until Google has crawled all the links? If you implemented this process you would never at one time have more than 300,000 links available for Google to crawl in sitemaps.
I read somewhere that eBay does something like this, it could be bogus info though.
Thanks
David
-
Thanks Maurizio.
What I am really most concerned about is submitting hundreds of sitemaps to Google and giving them concern that we might be spamming them.
This is why I am considering the second approach where we would submit 6 sitemaps at a time which would total no more than 300,000 links rather than giving them 200 plus sitemaps with 10 million links.
I should have been clearer in my reason for this question. The main goal here is to not have Google freakout because we just gave them 10,000,000 links at one time.
-
hI
it's better divide the sitemap in many files, max 50k and create
how you can read in this page
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=35738
"To fix this issue, break your Sitemap into several smaller Sitemaps, and list these in a Sitemap index file. (More information about Sitemap index files.) Upload your Sitemaps and Sitemap index files to your site, then submit these files individually."
Ciao
Maurizio
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Image Sitemap for non indexed Products
Hi, we have several ecommerce sites. We want to do an image sitemap, as we have lots of attractive images. The question is, can you put images for non-indexed products there as well, or does that conflict with the normal sitemap (the images would be indexed, the products not)? Thanks in advance. Dieter Lang
Intermediate & Advanced SEO | | Storesco0 -
2015/2016 Sitemaps Exclusions
Hello fellow mozrs!
Intermediate & Advanced SEO | | artdivision
Been working on a few Property (Real Estate for our American friends) websites recently and and two questions that constantly come up as we spec the site are: 1. What schema (schema.org) should the website use (throughout all pages as well as individual pages). Did anyone found that schema actually helped with their ranking/CTR?
2. Whilst setting up the sitemaps (usually Yaost is our preferred plugin for the job), what page would you EXCLUDE from the site map? Looking forward to some interesting comments.
Dan.0 -
Multiple Sitemaps Vs One Sitemap and Why 500 URLs?
I have a large website with rental listings in 14 markets, listings are added and taken off weekly if not daily. There are hundreds of listings in each market and all have their own landing page with a few pages associated. What is the best process here? I could run one sitemap and make each market's landing page .8 priority in the sitemap or make 14 sitemaps for each market and then have one sitemap for the general and static pages. From there, what would be the better way to structure? Should I keep all the big main landing pages in the general static sitemap or have them be at the top of the market segmented sitemaps? Also, I have over 5,000 urls, what is the best way to generate a sitemap over 500 urls? Is it necessary?
Intermediate & Advanced SEO | | Dom4410 -
XML Sitemaps settings for simple websites
We have numerous simple websites that are not updated very often (maybe once every 12-18 months). We are trying to create the perfect XML sitemap for them. Which brings me to my 2 questions:- 1. Should we include the "lastmod" tag in the XML sitemap? 2. What would you set the "changefreq" at? Once a month? Remember these sites are never updated as they are simple sites for industry sectors such as building and property maintenance. Please justify your answer and explain why as we are trying to understand.
Intermediate & Advanced SEO | | JohnW-UK0 -
Sitemap Issue - vol 2
Hello everyone! I validated the sitemap with different tools (w3Schools, and so on..) and no errors were found. So I uploaded into my site, tested it through GWT and BANG! all of a sudden there is a parsing error, which correspond to the last, and I mean last piece of code of thousand of lines, . I don't know why it isn't reading the code and it's giving me this as there are no other errors and I haven't got a clue about what to do in order to fix it! Thanks
Intermediate & Advanced SEO | | PremioOscar0 -
How Do I Create Multiple Pages In HTML Sitemap?
I'm working on an html sitemap for our ecommerce site and want to limit the links on each page to less than 100. I've created an article for the initial page, but what is the proper way to go to the next page? Do I create another article page (and so on and so on) until I have the sitemap completed? If so, how do I link from one page to the next? Would my on page text read: "sitemap continued" with anchor text on the link "sitemap page 2.."? It seems like all the sitemaps I've seen just fill one page with links and very little regard for "link saturation" and continuous pages. Thanks!
Intermediate & Advanced SEO | | AWCthreads0 -
301 redirect every pages?
Good evening, my question might sound stupid but please forgive me, I am still learning SEO. If I build a new site that will replace an existing site. Is there any point to do a 301 redirect for pages that had no inbound link so, no juice to pass? I kind of think that it would be a better practice to 301 redirect each pages to a page that make sense on the new web site .... but here is why I think that. Why I say that If I am lucky, many of my old web site pages will be indexed, many of them having no inbound links. So once the new web site online, until all my new web sites pages are indexed, I could imagine Google would send people to the index pages (the old ones that do not exist anymore)... I am right? So in that case, if I do a 301 redirect only for pages that have inbound links, the user would end up on a 404 page. Could you tell me if it make sense how I think? Thanks a lot !! Nancy P.S. I would not redirect if it make no sense to the user. I fully understand that we must always keep the user experience in mind in any 404 and 301 redirect decisions. But to simplify the question, just suppose it is ok from a user perspective to map every old site pages to a page in new web site.
Intermediate & Advanced SEO | | EnigmaSolution0 -
1 of the sites i work on keeps having its home page "de-indexed" by google every few months, I then apply for a review and they put it back up. But i have no idea why this keeps happening and its only the home page
1 of the sites i work on (www.eva-alexander.com) keeps having its home page "de-indexed" by google every few months, I then apply for a review and they put it back up. But i have no idea why this keeps happening and its only the home page I have no idea why and have never experienced this before
Intermediate & Advanced SEO | | GMD10