SEO for an exponentially growing site?
-
Hey Mozers!
I was having a quick chat with a friend the other day on doing SEO for a site that grows in page size at an exponential rate and was just wondering how you would go about optimizing it?
The example that we used would be a site that allowed users to upload videos and then have people vote on two videos against each other. So, if there are 100 uploaded videos and each of them are pared up with the other 99 to create a unique voting/battle page which has it's own unique URL, the site can get very large, VERY quickly. Meaning if just one more video is uploaded there would be
How exactly would you go about optimizing the site?
My biggest area of confusion would be generating sitemaps. I'm aware of best practices with large sitemaps (i.e. having a sitemap of sitemaps, not going over 50k in entries per sitemap etc..) But, how would you go about creating the sitemaps for this website if it's growing at an exponential rate, if at all?
If you have any other questions feel free to ask and I'll clarify it.
Thanks!
**TL;DR How would you optimize a site that grows at an exponential rate? **
-
Sorry, I wasn't very clear in what I initially responded back with.The code that I use doesn't create a new sitemap but rather just adds an additional node to the current one whenever a new page is added, so yes I am using the type of sitemap code that you are referring to.
BUT, since the site grows at an exponential rate it means that for every one video uploaded it will be matched up with each and every other single video to create a unique URL for each combination.
As an example if there are 10,000 videos currently on site which means there are 2 sitemaps right now and ONE more video is uploaded that means that 10,000 NEW unique urls/pages will be created thus creating 2 additional sitemaps resulting in a total of 4 sitemaps because of one new video being uploaded.
Let me know if I need to clarify anything.
-
Hello Jordan,
I am not sure what your using, but generally just because a video was uploaded to a site doesn't mean that it creates a new sitemap, for example in WordPress if you upload a video, it will get added to an existing sitemap. So when a search engine such as Google crawls it again, it will see the updated sitemap. So a automatic sitemap should work fine for you, unless your using a system that generates a new sitemap for everything uploaded, I have never heard of that, but I guess it's possible. If that's the case I am not sure at all. Very sorry I couldn't be of more assistance to you.
Have a great day and a happy holidays.
MB
-
Automatically generating sitemaps wasn't my concern it was more if you have 2 sitemaps worth of videos and then a user uploads just one more video that will end up creating an entire additional sitemap just because one more video was uploaded.
Thoughts?
-
Just as a note I would use dynamic meta tags that pull from titles, descriptions, for your meta info for each page so you don't run into duplicate meta issues. As for the sitemaps, that's a great question, I know there are many automated systems you can plugin to your site to auto generate sitemaps. I wish I knew more about them. I personally use WordPress for just about everything now, because it's easy to use and it has 1000's of useful plugins.
Have a great night.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I have an authority site with 90K visits per month. Now I have to change from non www to www. Will incur in any SEO issues while doing that? Could you please advice me on the best steps to follow to do this? Thank you very much!
Because I want to increase site speed, Siteground (my hosting) suggested I use Cloudflare Plus which needs my site to have www in order to work. I'm also using a cloud hosting. Im a bit scared of doing this, and thus decided to come to the community. I used MOZ for over 6 months now and love the tool. Please help me make the best possible decisions and what steps to follow. It would be much appreciated. Thank you!
Intermediate & Advanced SEO | | Andrew_IT0 -
SSL for SEO?
To obtain an SEO benefit from an SSL is there any particular type or brand which is recommended or has a track history? It seems you can pay anything between $20 and $???? (For that matter whatever you want to pay!). Any experience gratefully accepted! Thanks
Intermediate & Advanced SEO | | seoman100 -
SEO question regarding rails app on www.site.com hosted on Heroku and www.site.com/blog at another host
Hi, I have a rails app hosted on Heroku (www.site.com) and would much prefer to set up a Wordpress blog using a different host pointing to www.site.com/blog, as opposed to using a gem within the actual app. Whats are peoples thoughts regarding there being any ranking implications for implementing the set up as noted in this post on Stackoverflow: "What I would do is serve your Wordpress blog along side your Rails app (so you've got a PHP and a Rails server running), and just have your /blog route point to a controller that redirects to your Wordpress app. Add something like this to your routes.rb: _`get '/blog', to:'blog#redirect'`_ and then have a redirect method in your BlogController that simply does this: _`classBlogController<applicationcontrollerdef redirect="" redirect_to="" "url_of_wordpress_blog"endend<="" code=""></applicationcontrollerdef>`_ _Now you can point at yourdomain.com/blog and it will take you to the Wordpress site._
Intermediate & Advanced SEO | | Anward0 -
After Receiving a "Googlebot can't access your site" would this stop your site from being crawled?
Hi Everyone,
Intermediate & Advanced SEO | | AMA-DataSet
A few weeks ago now I received a "Googlebot can't access your site..... connection failure rate is 7.8%" message from the webmaster tools, I have since fixed the majority of these issues but iv noticed that all page except the main home page now have a page rank of N/A while the home page has a page rank of 5 still. Has this connectivity issues reduced the page ranks to N/A? or is it something else I'm missing? Thanks in advance.0 -
Site Navigation
Hi Mozzers, I am an SEO at uncommongoods.com and looking for your opinion on our site nav. Currently our nav & URLs are structured in 3 levels. From the top level down, they are: 1. Category ex: http://www.uncommongoods.com/home-garden 2. Subcat ex: http://www.uncommongoods.com/home-garden/bed-bath 3. Family ex:http://www.uncommongoods.com/home-garden/bed-bath/bath-accessories Right now, all levels are accessible from our top nav but we are considering removing the family pages. If we did that, Google could still find & crawl links to the family pages, but they would have to drill down to the subcat pages to find them. Do you guys think this would help or hurt our SEO efforts? Thanks! -Zack
Intermediate & Advanced SEO | | znotes0 -
Google penalized site--307/302 redirect to new site-- Via intermediate link—New Site Ranking Gone..?
Hi, I have a site that google had placed a manual link penalty on, let’s call this our
Intermediate & Advanced SEO | | Robdob2013
company site. We tried and tried to get the penalty removed, and finally gave up and purchased another name. It was our understanding that we could safely use either a 302 or 307 temporary redirect in order to redirect people from our old domain to our new one.. We put this into place several months and everything seemed to be going along well. Several days ago I noticed that our root domain name had dropped for our selected keyword from position 9 to position 65. Upon looking into our GWT under “Links to Your site” , I have found many, many, many links which were pointed to our old google penalized domain name to our new root domain name each of this links had a sub heading “Via this intermediate link -> Our Old Domain Google Penalized Domain Name” In light of all of this going on, I have removed the 307/302 redirect, have brought the
old penalized site back which now consists of a basic “we’ve moved page” which is linked to our new site using a rel=’nofollow’ I am hoping that -1- Our new domain has probably not received a manual penalty and is most likely now
received some sort of algorithmic penalty, and that as these “intermediate links” will soon disappear because I’m no longer doing the 302/307 from the old sight to the new. Do you think this is the case now or that I now have a new manual penalty place on the new
domain name.. I would very much appreciate any comments and/or suggestions as to what I should or can do to get this fixed. I need to still keep the old domain name as this address has already been printed on business cards many, many years ago.. Also on a side note some of the sub pages of the new root domain are still ranking very
well, it’s only the root domain that is now racking awfully.. Thanks,0 -
It appears that Googlebot Mobile will look for mobile redirects from the desktop site, but still use the SEO from the desktop site.
Is the above statement correct? I've read that its better to have different SEO titles & descriptions for mobile sites as users search differently on mobile devices. I've also read it's good to link build, keep text content on mobile sites etc to get the mobile site to rank. If I choose to not have titles & descriptions on my mobile site will Google just rank our desktop version & then redirect a user on a mobile device to our mobile site or should I be adding in titles & descriptions into the mobile site? Thanks so much for any help!
Intermediate & Advanced SEO | | DCochrane0 -
Critique My Site For SEO
Hi Everyone, I was wondering if someone might critique my site and let me know what you think. I've done pretty much everything I know to do proper seo for my site. I'd love to hear some critiques about what I am doing wrong. I'm not sure if my titles are okay, being that they are similar amongst pages. The other thing is that for all the javascript buttons on the top I have no followed them since they don't have any anchor text. The way google will crawl my page is through the links in the footer. I was thinking of moving them throughout the body of the page since I hear google isn't giving as much weight to footer links. I also wanted to hear what you think about putting a blog on my site and updating with fresh content as opposed to creating a separate blog and then linking back to my website with anchor text. Thanks for all the help. And glad to be a member Bill
Intermediate & Advanced SEO | | wsh150