350 (Out the 750) Internal Links Listed by Webmaster Tools Dynamically Generated-Best to Remove?
-
Greetings MOZ Community:
When visitors enter real estate search parameters in our commercial real estate web site, the parameters are somehow getting indexed as internal links in Google Webmaster Tools. About half are 700 internal links are derived from these dynamic URLs.
It seems to me that these dynamic alphanumeric URL links would dilute the value of the remaining static links.
Are the dynamic URLs a major issue? Are they high priority to remove?
The dynamic URLs look like this:
These URLs do not show up when a SITE: URL search is done on Google!
-
I believe your problem is in your robots.txt file. You're attempting a wildcard blocking of the search results pages with this line:
Disallow: /listings/search**?***
However, the asterisk ought to precede the question mark. If you want to block all URLs that include a question mark (?), do this:
Disallow: /listings/search***?**
Try that and see what happens. I've also found Aaron Wall's article on robots.txt to be helpful. Good luck!
Also, adding "noindex, nofollow" to the section does not necessarily keep a web page out of Google's index. When you think about it, you realize Google has to crawl the page the see that meta tag in the first place. Robots.txt is much stronger.
-
Hi Dennis:
The pages that display these search results are set to no-index, no-follow. They do not get indexed by Google in search results, just somehow register as about 200 links in internal links in Google Webmaster Tools.
How would I get these removed as links if they are no getting indexed by Google as pages? If I did get them removed is there a way of getting these links from being re-indexed?
I have attached an image of what these internal links look like in Google Webmaster Tools.
Thanks, Alan
-
I have seen both sides of the coin, some get affected, most don't.
I prefer to clean it up though and make sure those can't get indexed. I do this as part of our onpage SEO standards. We dont proceed until it's dealt with so it's a high priority.
It's cleaner, gets crawled easily and more efficient. Doesn't hurt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Poor internal linking?
Hi guys, Have a large e-commerce site 10,000 pages as a client and they are currently not getting much organic traffic to their level 3 sub-category pages, the URLs are like: https://www.domain.com.au/category/s...-category-type These pages have been on-page optimised, category content added, yet hardly any traffic. However the site level 1, level 2 pages do quite well. So this suggests this might be an internal linking issue? The site is definitely not penalized and as enough authority for these level 3 pages to rank. Any ideas would be very much appreciated! Cheers.
Intermediate & Advanced SEO | | bridhard80 -
What is the best way to add semantic linked data to WordPress?
As a recent Moz subscriber, I'm trying to up my game in terms of inbound marketing. One of the most pressing tasks is to add json-ld across all of my WordPress sites. What is the best way to do this? Should I use the technique set out here: https://moz.com/blog/using-google-tag-manager-to-dynamically-generate-schema-org-json-ld-tags Or should I use one of these plugins? https://en-gb.wordpress.org/plugins/schema/ https://en-gb.wordpress.org/plugins/wp-structuring-markup/ I want to get this right so any guidance would be gratefully received.
Intermediate & Advanced SEO | | treb0r0 -
Internal links to preferential pages
Hi all, I have question about internal linking and canonical tags. I'm working on an ecommerce website which has migrated platform (shopify to magento) and the website design has been updated to a whole new look. Due to the switch to magento, the developers have managed to change the internal linking structure to product pages. The old set up was that category pages (on urls domain.com/collections/brand-name) for each brand would link to products via the following url format: domain.com/products/product-name . This product url was the preferential version that duplicate product pages generated by shopify would have their canonical tags pointing to. This set up was working fine. Now what's happened is that the category pages have been changed to link to products via dynamically generated urls based on the user journey. So products are now linked to via the following urls: domain.com/collection/brand-name/product-name . These new product pages have canonical tags pointing back to the original preferential urls (domain.com/products/product-name). But this means that the preferential URLs for products are now NOT linked to anywhere on the website apart from within canonical tags and within the website's sitemap. I'm correct in thinking that this definitely isn't a good thing, right? I've actually noticed Google starting to index the non-preferential versions of the product pages in addition to the preferential versions, so it looks like Google perhaps is ignoring the canonical tags as there are so many internal links pointing to non-preferential pages, and no on-site links to the actual preferential pages? I've recommended to the developers that they change this back to how it was, where the preferential product pages (domain.com/products/product-name) were linked to from collection pages. I just would like clarification from the Moz community that this is the right call to make? Since the migration to the new website & platform we've seen a decrease in search traffic, despite all redirects being set up. So I feel that technical issues like this can't be doing the website any favours at all. If anyone could help out and let me know if what I suggested is correct then that would be excellent. Thank you!
Intermediate & Advanced SEO | | Guy_OTS0 -
Webmaster tools 404
Hey, I'm getting a soft 404 error on a webpage that has content and is deferentially not a 404. We've redirect a load of urls to the web page. The url has parameters which was used before the redirect but are no longer used on by the new url, these parameters have been carried over in the redirect. Is this whats causing the soft 404 error or is there another problem that may need addressing? Also a canonical has been set on the webpage. Thanks, Luke.
Intermediate & Advanced SEO | | NoisyLittleMonkey1 -
Internal links to blog posts
I am linking manually to blog posts in my site from my Home page. Our site isn't set up with an auto "Recent Posts" that shows on Home. Should I use the exact blog post title as the anchor text or do I need to create something that is not an exact match to the title of the post?
Intermediate & Advanced SEO | | gfiedel0 -
Are Navigation links different to static links
We are trying to reduce the number of links on our homepage. We could remove some fly out navigation links, We rank 1st on Google for some of these links. Would removing these hurt our SEO. The links are accessible 1 level down if we remove the homepage.
Intermediate & Advanced SEO | | Archers0 -
Best to Post Dynamic Content (Listings) under "Posts" in Wordpress?
My commercial real estate web site is being migrated to Wordpress from Drupal. Is it advisable to place dynamic content that will use taxonomy under "Posts" ? Listings will be changed every few months and there could be anywhere from several hundred to several thousand of them on the site. Developers have given me different advice. One has been adamant that listings and neighborhood pages (there will be about 25 neighborhood pages) should not be in the post section which is to be strictly reserved for blog entries. The last thing I want is to create a site structure which is unfriendly to SEO!!!! I would very much appreciate the perspective of anyone proficient with Wordpress and SEO. Thanks!!!
Intermediate & Advanced SEO | | Kingalan1
Alan Rosinsky0 -
10yr old Domain, Conflicting Message from Webmaster tools/Google search
This is the first time I have encountered this and am quite frankly a little baffled on how to proceed. We have some domains that are 10 years old, and do get some hits / impressions and they have a lot of content. So I redid the site in wordpress etc... Anyway, on Google the sites show up as www. , and on Webmaster tools,- the www. shows no impressions or anything, while the non-www domain shows up in google webmaster tools with data. The question is, if google displays the site as www. and webmaster tools shows data for non www. Which one do I proceed with, finding info on this has been pretty hard to do. Any input is appreciated, Thanks in advance:)
Intermediate & Advanced SEO | | choiceenergy0