SEO for an exponentially growing site?
-
Hey Mozers!
I was having a quick chat with a friend the other day on doing SEO for a site that grows in page size at an exponential rate and was just wondering how you would go about optimizing it?
The example that we used would be a site that allowed users to upload videos and then have people vote on two videos against each other. So, if there are 100 uploaded videos and each of them are pared up with the other 99 to create a unique voting/battle page which has it's own unique URL, the site can get very large, VERY quickly. Meaning if just one more video is uploaded there would be
How exactly would you go about optimizing the site?
My biggest area of confusion would be generating sitemaps. I'm aware of best practices with large sitemaps (i.e. having a sitemap of sitemaps, not going over 50k in entries per sitemap etc..) But, how would you go about creating the sitemaps for this website if it's growing at an exponential rate, if at all?
If you have any other questions feel free to ask and I'll clarify it.
Thanks!
**TL;DR How would you optimize a site that grows at an exponential rate? **
-
Sorry, I wasn't very clear in what I initially responded back with.The code that I use doesn't create a new sitemap but rather just adds an additional node to the current one whenever a new page is added, so yes I am using the type of sitemap code that you are referring to.
BUT, since the site grows at an exponential rate it means that for every one video uploaded it will be matched up with each and every other single video to create a unique URL for each combination.
As an example if there are 10,000 videos currently on site which means there are 2 sitemaps right now and ONE more video is uploaded that means that 10,000 NEW unique urls/pages will be created thus creating 2 additional sitemaps resulting in a total of 4 sitemaps because of one new video being uploaded.
Let me know if I need to clarify anything.
-
Hello Jordan,
I am not sure what your using, but generally just because a video was uploaded to a site doesn't mean that it creates a new sitemap, for example in WordPress if you upload a video, it will get added to an existing sitemap. So when a search engine such as Google crawls it again, it will see the updated sitemap. So a automatic sitemap should work fine for you, unless your using a system that generates a new sitemap for everything uploaded, I have never heard of that, but I guess it's possible. If that's the case I am not sure at all. Very sorry I couldn't be of more assistance to you.
Have a great day and a happy holidays.
MB
-
Automatically generating sitemaps wasn't my concern it was more if you have 2 sitemaps worth of videos and then a user uploads just one more video that will end up creating an entire additional sitemap just because one more video was uploaded.
Thoughts?
-
Just as a note I would use dynamic meta tags that pull from titles, descriptions, for your meta info for each page so you don't run into duplicate meta issues. As for the sitemaps, that's a great question, I know there are many automated systems you can plugin to your site to auto generate sitemaps. I wish I knew more about them. I personally use WordPress for just about everything now, because it's easy to use and it has 1000's of useful plugins.
Have a great night.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Quick SEO Audit of my site.
Hello, I hope you are doing great. I am working on a website that is related to flea collars for cats and dogs. And I want you to make a quick audit of the site where am I lacking. It could be great if you can help ASAP. You can view my site here :
Intermediate & Advanced SEO | | Request4peace0 -
SEO Impact & Google Impact On Removing Product From Category Page for Ecommerce Site
Hello Experts, For my Ecommerce site previously I was showing products at category pages i.e. first all subcategories name after that list all products of all subcateogries. That also approx per category 500 products via load more feature. My query is now I am planning to show products only at Product Listing Page and not on Category pages so what will be SEO impact and how google will treat this? Thanks!
Intermediate & Advanced SEO | | Johny123450 -
Mobile First Index: What Could Happen To Sites w Large Desktop but Small Mobile Sites?
I have a question about how Mobile First could affect websites with separate (and smaller) mobile vs desktop sites. Referencing this SE Roundtable article (seorountable dot com /google-mobile-first-index-22953.html), "If you have less content on your mobile version than on your desktop version - Google will probably see the less content mobile version. Google said they are indexing the mobile version first." But Google/ Gary Illyes are also on the record stating the switch to mobile-first should be minimally disruptive. Does "Mobile First" mean that they'll consider desktop URLs "second", or will they actually just completely discount the desktop site in lieu of the mobile one? In other words: will content on your desktop site that does not appear in mobile count in desktop searches? I can't find clear answer anywhere (see also: /jlh-marketing dot com/mobile-first-unanswered-questions/). Obviously the writing is on the wall (and has been for years) that responsive is the way to go moving forward - but just looking for any other viewpoints/feedback here since it can be really expensive for some people to upgrade. I'm basically torn between "okay we gotta upgrade to responsive now" and "well, this may not be as critical as it seems". Sigh... Thanks in advance for any feedback and thoughts. LOL - I selected "there may not be a right answer to this question" when submitting this to the Moz community. 🙂
Intermediate & Advanced SEO | | mirabile0 -
Old site penalised, we moved: Shall we cut loose from the old site. It's curently 301 to new site.
Hi, We had a site with many bad links pointing to it (.co.uk). It was knocked from the SERPS. We tried to manually ask webmasters to remove links.Then submitted a Disavow and a recon request. We have since moved the site to a new URL (.com) about a year ago. As the company needed it's customer to find them still. We 301 redirected the .co.uk to the .com There are still lots of bad links pointing to the .co.uk. The questions are: #1 Do we stop the 301 redirect from .co.uk to .com now? The .co.uk is not showing in the rankings. We could have a basic holding page on the .co.uk with 'we have moved' (No link). Or just switch it off. #2 If we keep the .co.uk 301 to the .com, shall we upload disavow to .com webmasters tools or .co.uk webmasters tools. I ask this because someone else had uploaded the .co.uk's disavow list of spam links to the .com webmasters tools. Is this bad? Thanks in advance for any advise or insight!
Intermediate & Advanced SEO | | SolveWebMedia0 -
When Mobile and Desktop sites have the same page URLs, how should I handle the 'View Desktop Site' link on a mobile site to ensure a smooth crawl?
We're about to roll out a mobile site. The mobile and desktop URLs are the same. User Agent determines whether you see the desktop or mobile version of the site. At the bottom of the page is a 'View Desktop Site' link that will present the desktop version of the site to mobile user agents when clicked. I'm concerned that when the mobile crawler crawls our site it will crawl both our entire mobile site, then click 'View Desktop Site' and crawl our entire desktop site as well. Since mobile and desktop URLs are the same, the mobile crawler will end up crawling both mobile and desktop versions of each URL. Any tips on what we can do to make sure the mobile crawler either doesn't access the desktop site, or that we can let it know what is the mobile version of the page? We could simply not show the 'View Desktop Site' to the mobile crawler, but I'm interested to hear if others have encountered this issue and have any other recommended ways for handling it. Thanks!
Intermediate & Advanced SEO | | merch_zzounds0 -
SEO Recommendations
For about 3 years our website was number one in Google.co.uk for our trades main keyphrase which resulted in excellent sales. About 12 months ago that position started to slide downwards and for that keyphrase we are now number 10. We are still at number's 1,2 and 3 for several other keyphrases but ones that result in fewer daily Google searches and resultant sales. I have always added unique content to the site but admit that my blog posts became less than daily over the past 12 months. However I am adding posts as often as I can now, of good length and of unique content. As well as tweaking all our online seo factors I'm trying to add good backlinks as often as possible. I wonder if anyone has been in a similar position and what they did to try and regain their previous position? Colin
Intermediate & Advanced SEO | | NileCruises0 -
Redirection to mobile site
Calling all SEO ninjas! I'm currently developing single web pages for various clients which function as abbreviated versions of their main websites. They are all related & under a single domain. When a user visits these pages on a mobile device, CSS is used to display mobile friendly versions of these pages. My clients are thrilled with these mobile versions and now want to also redirect mobile visitors from their main site (which is not mobile optimised) to these pages. My questions are: Are there any negative implications if we did this? ie. redirecting to a different domain What is the best method for redirection? eg. JavaScript Can this be achieved by adding a single line of code to their main site Can this be done in an SEO friendly way so that the redirection acts like a backlink? Many thanks.
Intermediate & Advanced SEO | | martyc0 -
Linking to bad sites
Hi, I just have a quick question. Is it very negative to link to "bad" sites, such as online pharmacies, dating, adult sites, that sort of stuff? How much does linking to a "bad" site negatively affect a "good" site? Thank you.
Intermediate & Advanced SEO | | salvyy0