Getting pages that load dynamically into the SE's
-
SEO'ers,
Am dealing with an issue I cannot figure out the best way to handle. Working on a website that shows the definitions of words which are loaded dynamically from an open source. Source such as: wiktionary.org
When you visit a particular page to see the definition of the word, say; www.example.com/dictionary/example/ the definition is there. However, how can we get all the definition pages to get indexed in search engines? The WordPress sitemap plugin is not picking up these pages to be added automatically - guess because it's dynamic - but when using a sitemap crawler pages are detected.
Can anybody give advice on how to go about getting the 200k+ pages indexed in the SE's? If it helps, here's a reference site that seems to load it's definitions dynamically and has succeeded in getting its pages indexed: http://www.encyclo.nl/begrip/sample
-
I see what you mean there - thanks for sharing your expertise and views on this issue. Much appreciated
-
The only way I'd let those pages be indexed is if they had unique content on them AND/OR provided value in other ways besides just providing the Wiki definition. There are many possibilities for doing this, none of them scalable in an automated fashion, IMHO.
You could take the top 20% of those pages (based on traffic, conversions, revenue...) and really customize them by adding your own definitions and elaborating on the origin of the word, etc... Beyond that you'd probably see a decline in ROI.
-
Everett, yes that's correct. I will go ahead and follow up on what you said. I do still wonder what the best way would be to go about getting it indexed - if I wanted to do that in the future. If you could shed some light on how to go about that, I'd really appreciate it. Thanks so much in advance!
-
It appears that your definitions are coming from wiktionary.org and are therefore duplicate content. If you were providing your own definitions I would say keep the pages indexable, but in this case I would recommend adding a noindex, follow robots meta tag to the html header of those pages.
-
Hi Everett, I've been looking at the index for word definitions and there's so many pages that are very similar to each other. It's worth giving it a shot I think. If you can provide feedback please do. Here's the domain: http://freewordfinder.com. The dictionary is an addition to users who'd like to see what a word means after they've found a word from random letters. You can do a search at the top to see the results, then click through to the definition of the word. Thanks in advance
-
Ron,
We could probably tell you how to get those pages indexed, but then we'd have to tell you how to get them removed from the index when Google sees them all as duplicate content with no added value. My advice is to keep them unindexed, but if you really want them to be indexed tell us the domain and I'll have a look at how it's working and provide some feedback.
-
Hi Keri, did you think that the site might get penalized because it would in essence be duplicate content from another site? Even though the source is linked from the page? Please let me know your thoughts when you can
-
No they currently do not have additional information on them. They are simply better organized on my pages compared to the 3rd party. The unique information is what drives visitors to the site and from those pages it links to the definitions just in case they're interested understanding the meaning of a word. Does that help?
-
Do the individual pages with the definitions have additional information on them, or are they just from a third party, with other parts of the site having the unique information?
-
Hi Keri, thanks for your response. Well, I see what you're saying. The pages that show the definition pulled from the 3rd party are actually supplementary to the solution the site provides (core value). Shouldn't that make a difference?
-
I've got a question back for you that's more of a meta question. Why would the search engines want to index your pages? If all the page is doing is grabbing information from another source, your site isn't offering any additional value to the users, and the search engine algos aren't going to see the point in sending you visitors.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does not having any hreflang tags for U.S Visitors lead to an increase in International Visitors?
I have seen a massive increase in International Visitors on our website and visitors within the United States dropped off hard this month (by about 20%). Could it be possible that not having any hreflang tags can lead to an increase in International Customers visiting the site even though your sitemap is set to "Target users in United States" within the Google Search Console? In the Google Search Console, I have International Targeting set to "Target users in United States." However, Google Search Console is saying our site doesn't have any hreflang tags. In the Google Search Console, it says "Your site has no hreflang tags. Google uses hreflang tags to match the user's language preference to the right variation of your pages." I'm not sure when that was flagged, but recently we have seen a massive increase in International Visitors to our site from countries such as Russia, Vietnam, Indonesia, the United Kingdom and so on. This poses a problem since our chances of turning one of those visitors into a customer is extremely slim. Along with that, nearly every international customer is contributing to an extremely high Bounce Rate. Attached is a screenshot of the Error about hreflang tags. https://imgur.com/a/XZI45Pw And here is a screenshot of the Country we are targeting. https://imgur.com/a/ArpWe9Z Lastly, attached is a screenshot of all of the Countries that visited our site today: https://imgur.com/a/d0tNwkI
International SEO | | MichaelAtMSP1 -
How should I handle hreflang tags if it's the same language in all targeting countries?
My company is creating an international version of our site at international.example.com. We are located in the US with our main site at www.example.com targeting US & Canada but offering slightly different products elsewhere internationally. Ideally, we would have hreflang tags for different versions in different languages, however, it's going to be an almost duplicate site besides a few different SKUs. All language and content on the site is going to be in English. Again, the only content changing is slightly different SKUs, they are almost identical sites. The subdomain is our only option right now. Should we implement hreflang tags even if both languages are English and only some of the content is different? Or will having just canonicals be fine? How should we handle this? Would it make sense to use hreflang this way and include it on both versions? I believe this would be signaling for US & Canda visitors to visit our main site and all other users go to the international site. Am I thinking this correctly or should we be doing this a different way?
International SEO | | tcope250 -
Geo-Targeting separate TLD's where both are .com domains
Hi I have a client who owns two separate TLDs for the same brand (for the sake of this post, we'll call the two sites www.site-a.com and www.site-b.com). For site www.site-a.com the website has been around for a while and is their primary site for their US operations which is their heartland, is well established in the SERPS and is where they make most of their money. As they looked to expand to the UK, they then created www.site-b.com and added the UK as a subfolder (so www.site-b.com/uk) and geo-targeted it towards the UK in Webmaster tools . The site has recently launched but they now find that, when a customer searches for their brand in the UK, they find www.site-a.com in position 1 (which, given it's tailored for a primary US audience, has a significantly lower conversion rate for UK traffic) and www.site-b.com in position 2. However, the client doesn't want to specifically geo target www.site-a.com to the USA as they feel it might affect where they appear for other international markets aside from the UK. So the question is, how can they, with the existing infrastructure, help remove www.site-a.com from the UK SERPs without adversely affecting their rank elsewhere? Hope this makes sense and thanks in advance for your help. James
International SEO | | jimmygs19820 -
I have more than 4000 pages but still have a low trafic. Would love to know more to be better ranked ?
My website is a magazine about travel and fashion. But even if i have a lot of pages, I am still low in ranking. Why ? Thanks for any advice !
International SEO | | ccjourn0 -
Include mobile and international versions of pages to sitemap or not?
My pages already have alternate and hreflang references to point to international and mobile versions of the content. If I add 5 language desktop versions and 5 language mobile versions as https://support.google.com/webmasters/answer/2620865?hl=en explains, my sitemap will get bulky. What are the pros and cons for referencing all page versions in sitemap and for include just general (English/Desktop) version in sitemap?
International SEO | | poiseo0 -
Subfolders and 301's
Hello all, Quite simply, I'm stuck. Well, I think I am. We are about to launch a whole new International side of our website. We're an education job board www.eteach.com for schools in the UK and a little internationally. Now that the business is growing we want to make our brand more global. All the big bosses wanted to create a brand new website called www.eteachinternational.com. I managed to persuade them to not to do that and instead use a subfolder approach off of our well established and strong domain www.eteach.com (phew). However, now I'm getting a little lost in making sure I don't duplicate my content. We have a staffroom section on our website which basically has lots of relevant content for people searching how to become a teacher, e.g. www.eteach.com/how-to-become-a-teacher. We also want this same content on the international subfolder, as it will still be relevant content for international teachers. However... Do I have to completely re-write the content (which I'm trying to avoid as it will be very similar) or can I put in a rel=canonical to the already existing pages? So basically (I know this HTML isn't right, it's just for visual's sake!): www.eteach.com/international/how-to-become-a-teacher rel=canonical --> www.eteach.com/how-to-become-a-teacher I understand this gives all the authority to the original page, not the international one, but I'm fine with that (unless anyone can suggest anything else?)
International SEO | | Eteach_Marketing0 -
Why don't our English versions show up first?
If I google "greatfire" I find the Chinese version of our website (zh.greatfire.org) before the English version (en.greatfire.org). This is not on the Chinese-language version of Google. Why is this? Our site even has a language indicator () and also hints of where the English version is (). The same thing happens if I google "freeweibo". I find https://freeweibo.com but not https://freeweibo.com/en/, even though we indicate that's the English version (). Any ideas?
International SEO | | GreatFire.org0 -
How can I rank couple of pages to a specific geography ?
Hi guys, I have a pretty good success in many of my keyword on google US. We are a multi-country company and would like to get better ranking on all these countries. I know it's a long run and we need to by patient to get the rank desired. We are getting the slowly, bu surely. In the next couple of months, we will be attending a conference where we will have a booth and we would like to conduct a campaign to invite customers to join us. My question is : Is there an efficient way to have just couple of pages on our web site that could potentially rank fast on a specific geography ? Europe is my target audience ( France an UK ). If you have any advice, I would appreciate. Best regards,
International SEO | | processia1