Should our social network put all of our member profiles in the site map?
-
We're working on some changes to our site and want to add a site map file to give some hints to search engines on our member profiles. Our site has millions of member profiles. I want to ensure that Google indexes them.
Is there a benefit to submitting millions of profiles across multiple site maps?
Is it better to pick a sample of those and submit them? Most member profiles have links to other member profiles.
Is there any benefit to taking a single site map of 50k links and then changing it at some frequency so that all of the member profiles are eventually indexed?
Thanks.
-
This was the approach we were considering. However, what gave me pause was looking at other social networks are doing.
- LinkedIn only has 50k profiles.
- Meez has multiple index files and looks like they are indexing many of their profiles
- Facebook and MySpace don't appear to use Sitemaps (although they do have a crawlable member directory)
- WeeWorld has 500 links
The point is there doesn't seem to be any consistency.
If we end up creating a human readable directory ala Facebook does that achieve the same thing?
-
Totally agree with mike and john.
-
I would definitely do this. Since you have so many profiles, you'll want to have one main sitemaps linking to smaller sitemaps with the profile URLs. One of the sites I work on has several million activities created by users that we put into sitemaps. We automatically generate our sitemaps files frequently and add the new activities created. We include a last modified date for each activity as well so the search bot will know if anything has changed since the last time it's indexed it.
I would create a sitemap system where all of your profiles could be found, and by including the last modified date, you can leave it up to the searchbot as to whether or not a profile has been updated and needs to be re-indexed. There are a couple other properties you could use listed on http://www.sitemaps.org/protocol.php as well.
-
Yes, we have the concept of public / private profiles. Private profiles aren't included in the index.
-
Benefiting or not, make sure users have the ability to turn this off to avoid privacy problems.
It might create a mess though if they link back and forth.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Consolidating a Large Site with Duplicate Content
I will be restructuring a large website for an OEM. They provide products & services for multiple industries, and the product/service offering is identical across all industries. I was looking at the site structure and ran a crawl test, and learned they have a LOT of duplicate content out there because of the way they set up their website. They have a page in the navigation for “solution”, aka what industry you are in. Once that is selected, you are taken to a landing page, and from there, given many options to explore products, read blogs, learn about the business, and contact them. The main navigation is removed. The URL structure is set up with folders, so no matter what you select after you go to your industry, the URL will be “domain.com/industry/next-page”. The product offerings, blogs available, and contact us pages do not vary by industry, so the content that can be found on “domain.com/industry-1/product-1” is identical to the content found on “domain.com/industry-2/product-1” and so-on and so-forth. This is a large site with a fair amount of traffic because it’s a pretty substantial OEM. Most of their content, however, is competing with itself because most of the pages on their website have duplicate content. I won’t begin my work until I can dive in to their GA and have more in-depth conversations with them about what kind of activity they’re tracking and why they set up the website this way. However, I don’t know how strategic they were in this set up and I don’t think they were aware that they had duplicate content. My first thought would be to work towards consolidating the way their site is set up, so we don’t spread the link-equity of “product-1” content, and direct all industries to one page, and track conversion paths a different way. However, I’ve never dealt with a site structure of this magnitude and don’t want to risk messing up their domain authority, missing redirect or URL mapping opportunities, or ruin the fact that their site is still performing well, even though multiple pages have the same content (most of which have high page authority and search visibility). I was curious if anyone has dealt with this before and if they have any recommendations for tackling something like this?
On-Page Optimization | | cassy_rich0 -
Help finding someone to handle crashing site/site optimization.
I need someone who can handle website/WordPress issues as they come up.For example, my site has gone down 4 times tonight, and my host can't figure it out. They also keep recommending that I optimize my site, but I don't know how. I need a go-to web person for this sort of thing. Any recommendations?
On-Page Optimization | | cbrant7770 -
How do I reduce the amount of internal links on my site?
Hi, Can someone help me with reducing the amount of internal links on our site please? https://www.thepresentfinder.co.uk Thanks Charlie
On-Page Optimization | | The-Present-Finder0 -
Do sites with more pages rank better?
If a site has more pages and also has good internal linking, then logically it would rank better. Is this the case? Should I be building big (but high quality) sites?
On-Page Optimization | | T0BY0 -
Would it be inappropriate use of authorship mark-up for a directory profile?
If a business creates a profile in a directory and adds their Google+ page, and they add the domain/profile URL as a publisher they contribute to, is that a legit use? They are the author of the business profile, but it's not an article.
On-Page Optimization | | designquotes0 -
Article on site and distribution, is it duplicate content?
I was always taught to place all original articles on site, let them get indexed by Google, then put out for distribution through various press release outlets. With the latest penguin update, how does this practice work out concerning duplicate content? In theory, I wrote the article so I should get credit for it on my site first, then push through various distribution outlets to get it out to my targeted audience in my niche field. Typing out loud I would tend to think if the article is on my site first then I would get credit and any others following would be hit by duplicate content if in fact google considered it a dupe violation. Any input on this? Am I on track or am I heading for a train wreck.
On-Page Optimization | | anthonytjm0 -
Sudden Site Rankings Drop
Good day guys, We have been following strict SEO strategies for the past 6 months, all sites have been improving incredibly well, all except one. The site in question is http://bit.ly/IH4pkM . The site is regarding automotive spray booth equipment. We were ranking on the first page for the keyword "spray booth" (which is the most important one), at place #4 for weeks on end. However since half-way last week, the site has been dropped to half-way the second page (#17). There are barely any crawler errors listed for our campaign on SEOMoz. There were several pages of which the meta description was missing, but that has been fixed earlier this week. When it comes to link building, I looked at what the top competitors were doing, and was looking for unique link building opportunities myself. We have received 0 webmaster tools warnings as well. I do not believe we are penalized due to the "penguin" update. After all, if you search for for the company's name in Google, it is still listed on there (# 2). Nor have we been part of dodgy link networks at all. So my question is, what do you guys believe made us drop the rankings? Is there some on-page issues I am overlooking? Any recommendations to restore out previous rankings? Kind Regards, Roderic
On-Page Optimization | | Michael-Goode0 -
Best information organization for a new site?
I'm launching a new stain removal website, and wanted to know what would be considered the best way to organize the content? Since most articles will roughly involve "removing X from Y" or "how to remove Z," I can see two ways... 1. Organize articles by Stained Items, Stain Agents and perhaps Cleaning Detergents. 2. Spread the categories out more, to try and group stained items according to categories... E.g. Hard surfaces, delicates, fabrics, ceramics etc. Any thoughts on which of these two might be the best way to organize the site, or are there any better suggestions? Not sure what the main considerations are here... Either of these two seem equally user-friendly.
On-Page Optimization | | ZakGottlieb710