350 (Out the 750) Internal Links Listed by Webmaster Tools Dynamically Generated-Best to Remove?
-
Greetings MOZ Community:
When visitors enter real estate search parameters in our commercial real estate web site, the parameters are somehow getting indexed as internal links in Google Webmaster Tools. About half are 700 internal links are derived from these dynamic URLs.
It seems to me that these dynamic alphanumeric URL links would dilute the value of the remaining static links.
Are the dynamic URLs a major issue? Are they high priority to remove?
The dynamic URLs look like this:
These URLs do not show up when a SITE: URL search is done on Google!
-
I believe your problem is in your robots.txt file. You're attempting a wildcard blocking of the search results pages with this line:
Disallow: /listings/search**?***
However, the asterisk ought to precede the question mark. If you want to block all URLs that include a question mark (?), do this:
Disallow: /listings/search***?**
Try that and see what happens. I've also found Aaron Wall's article on robots.txt to be helpful. Good luck!
Also, adding "noindex, nofollow" to the section does not necessarily keep a web page out of Google's index. When you think about it, you realize Google has to crawl the page the see that meta tag in the first place. Robots.txt is much stronger.
-
Hi Dennis:
The pages that display these search results are set to no-index, no-follow. They do not get indexed by Google in search results, just somehow register as about 200 links in internal links in Google Webmaster Tools.
How would I get these removed as links if they are no getting indexed by Google as pages? If I did get them removed is there a way of getting these links from being re-indexed?
I have attached an image of what these internal links look like in Google Webmaster Tools.
Thanks, Alan
-
I have seen both sides of the coin, some get affected, most don't.
I prefer to clean it up though and make sure those can't get indexed. I do this as part of our onpage SEO standards. We dont proceed until it's dealt with so it's a high priority.
It's cleaner, gets crawled easily and more efficient. Doesn't hurt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirect wordpress.com and internal link ?
Hi Moz Fans, First of all, I need to say thanks to all of answer to previous post. And today i also have the another question that similar to that post. Because our website using Wordpress.org as our CMS for blog post then easier to redirect by point to new site, According to setting site URL ? However in our each blog articles also have anchor text as internal link that link to another blog post, Which mean those link will be automatic redirect to new URL. So once Google bot re-crawl our website when we tell the Google by webmaster tools and the redirection we using 301. What will be happen when Google Bot crawl those link again We need to changes those link as well Keep same with redirection. Nothing happen
Intermediate & Advanced SEO | | ASKHANUMANTHAILAND0 -
Bypassing Google, Data Highlighter and Webmaster tools
eLLo! Has anyone used Data Highlighter? I've had colleagues mentioning a jump in CTR after using the data highlighter on pages. Thought I'll do the same and went into my webmaster tools but I've hit a brick wall. Whenever I highlight a product page, my country selector pops up and I'm unable to highlight a product page. A colleague of mine mentioned to bypass google by basing it on user agent, this will allow you to avoid the country selector. But if I bypass Google, wouldn't it affect Google Analytics, Indexing etc?
Intermediate & Advanced SEO | | Bio-RadAbs0 -
Is it dangerous to use "Fetch as Google" too much in Webmaster Tools?
I saw some people freaking out about this on some forums and thought I would ask. Are you aware of there being any downside to use "Fetch as Google" often? Is it a bad thing to do when you create a new page or blog post, for example?
Intermediate & Advanced SEO | | BlueLinkERP0 -
If it's not in Webmaster Tools, is it Duplicate Title
I am showing a lot of errors in my SEOmoz reports for duplicate content and duplicate titles, many of which appear to be related to capitalization vs non-capitalization in the URL. Case in point, if a URL contains a lower character, such as: http://www.gallerydirect.com/art/product/allyson-krowitz/distinct-microstructure-i as opposed to the same URL having an upper character in the structure: http://www.gallerydirect.com/art/product/allyson-krowitz/distinct-microstructure-I I am finding that some of the internal links on the site use the former structure and other links use the latter structure. These show as duplicate title/content in the SEOmoz reports, but they don't appear as duplicate titles in Webmaster Tools. My question is, should I try to work with our developers to create a script to change all of the content with cap letters in the destination links internally on the site, or is this a non-issue since it doesn't appear in Webmaster Tools?
Intermediate & Advanced SEO | | sbaylor0 -
How do I find the links on my site that link to another one of my pages?
I ran IIS Seo toolkit and it found about 40 pages that I have no idea how they exist. What tool can I use to find out what internal link is linking to them so I can fix them or get rid of them?
Intermediate & Advanced SEO | | EcommerceSite0 -
Why do pages with a 404 error drop out of webmaster tools only to reappear again?
I have noticed a lot of pages which have fallen out of webmaster tools crawl error log that had bee 404'ing are reappearing again Any suggestions as to why this might be the case? How can I make sure they don't reappear again?
Intermediate & Advanced SEO | | Towelsrus0 -
301'ing over 700 internal links to the main page
I just got a contract for a site. After I analyzed their website, I noticed that they have over 700 pages indexed. However, their internal linking structure sucks. It's basically all 700 pages in one directory. What do you recommend? I redirect all the internal structures to their new locations, or would it be better to redirect all those internal pages to their main domain name, and build a completely new seo-friendly structure? Redirecting their current pages to each individual page is gonna take a lotta time, and I don't think they're gonna pay for it. :l
Intermediate & Advanced SEO | | skgppa0 -
There's a website I'm working with that has a .php extension. All the pages do. What's the best practice to remove the .php extension across all pages?
Client wishes to drop the .php extension on all their pages (they've got around 2k pages). I assured them that wasn't necessary. However, in the event that I do end up doing this what's the best practices way (and easiest way) to do this? This is also a WordPress site. Thanks.
Intermediate & Advanced SEO | | digisavvy0
