How to detect where Google gets indexed URL's
-
Google index some kind of way some links that create duplicate content. We doesn't understand how these are created so we would like detect where Google robots find these links.
We tried:
- Moz Crawl Diagnostics but it shows 0 as Internal Link Count for these kind of links.
- Find some information from Google Analytics, that maybe there is trace (site content - all content) from visitors side. There wan't.
- We tried to find some information in Webmaster Tools under Internal link and HTML Improvements but didn't find any trace.
- Tried some search commands. Is there maybe some good one to search.
- TO search URL's form code with https://search.nerdydata.com.
-
It really isn't possible for an outsider to know why your website is generating those URLs in error; you would have to talk to your developer about that.
As far as canonicals, if your problem is page.com is getting duplicated by added parameters: page.com/?id=1, page.com/?id=2, page.com/?id=3, etc. as long as you have the canonical on page.com, all of the parameter pages will have the correct canonical on them as well. (But you are right, you should track down the source; your developer will know.)
-
Thanks you for your answer but yes I know that these are generated by our site. But problem is that I can use canonical tag for these that are indexed right now but later new ones will be created someway. Problem root isn't that we doesn't know how to use canonical, it's how to get to know where these URL's are find/indexed/detected by Google.
These kind of URL's have been there for months so we can't just hope that somehow these will be droped. We need to find some kind of solution and detect real problem.
-
If you found those URLs by doing a site: search, then those parameters are being generated by your site. (I am surprised that Google is even indexing them; I assume that pretty soon all but one will be dropped.) Here is an article that explains more about those types of duplicate pages: http://moz.com/blog/which-page-is-canonical
You can fix this by using a canonical tag on your homepage with the version that doesn't have the parameter.
-
Our front page has almost 50 duplicate versions. These are shown when we do site:oursite.com, there are /et?id=xx, /et?productId=xx, etc. In URL xx are different numbers.
-
Where are you seeing these duplicate content links? Does Webmaster Tools say that they are duplicate content? Or does this show up in your Moz crawl? What do these URLs look like?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google My Business app search report
What is the deal with the search reports in the Google My Business app? I downloaded this app so prospective customers could message my business, and when I look at the search reports on the app, the results seem nonsensical. According to google analytics my business receives pretty steady traffic every day. Why does the report say that I receive zero visitors one day and 400 the next? (See the screenshot below) 4yLRCHG
Reporting & Analytics | | RandyHT1 -
Google Analytics Not Properly Attributing Goals
My GA has been working fine for years and is now suddenly attributing 100% of my goals/conversions to a "referral" source. In this case "crm.zoho.com". (see attached url) Our contact form is a zoho CRM form. When prospect fills it out online, it dumps them to a thank-you page and the conversion is counted. That all works just fine, but it is not attributing the conversion to Organic or PPC or Direct as it was a month or so ago. I'm not sure if this may be the cause, but I was cleaning up GA about a month ago, deleting some filters I didn't think I needed any more. Thank you for your help. 9CRUx
Reporting & Analytics | | sanctuary2420 -
No-indexed pages are still showing up as landing pages in Google Analytics
Hello, My website is a local job board. I de-indexed all of the job listing pages on my site (anything that starts with http://www.localwisejobs.com/job/). When I search site:localwisejobs.com/job/, nothing shows up. So I think that means the pages are not being indexed. When I look in Google Analytics at Acquisition > Search Engine Optimization > Landing Pages, none of the job listing pages show up. But when I look at Acquisition > Channels > Organic and then click Landing Page as the primary dimension, the /job pages show up in there. Why am I seeing this discrepency in Organic Landing pages? And why would the /job pages be showing up as landing pages even though they aren't indexed?
Reporting & Analytics | | mztobias0 -
Google Webmaster Tools During GA Transition?
I'm working with a client that is launching a new website. Google Webmaster Tools can just be disconnected, then reconnected to the new Google Analytics property, correct? Without any data loss in Webmaster Tools? Thanks! Becky
Reporting & Analytics | | Becky_Converge0 -
Exact Match in Google Search (Not Adwords)
I was going throught the list of keywords that have sent traffic to my site over the last 7 years and cam across one "A516 grade 70" that had hundreds of variants. Now in a lot of cases search volumes were different as were SERPS. We've tested a few variants with reworked pages (70% similar to original but optimised for variant keyword) and see good SERPS and traffic results. Theres obviously some diminishing returns here for us but the interesting question is when to these variants become an exact match and when not? In some cases the variants are unique because of the spacig, periods and hyphens used. there isn't a clear correlation with exact matc though. Insight appreciated. (Sorry for spelling errors. Form doesn't play nicely with iPad)
Reporting & Analytics | | Zippy-Bungle0 -
Big variation in the number of search results. (person's name)
Hi, I have been noticing a really dramatic variation in the number of results Google is returning for the name "Carolyn Hadlock." Most of the time it seems to be around 2000. But then it will jump up to over 10,000. Does anyone know why there would be such a big jump? And then why it would go back? If tested both logged into Google and then not - as well as having others log is as themselves. That does not seem to be it. Any thoughts would be much appreciated.
Reporting & Analytics | | yandl0 -
How to filter many cities from Google analytics
I need to filter hundreds of cities from my traffic reports. I've been told that adding them all in this format: |Knutsford|Ripon|Cheadle Hulme|Congleton| and using the 'matching reg/exp from the drop down should do it... Unfortunately that doesn't seem to be working as I can still see cities like London in the results.... Any ideas?
Reporting & Analytics | | david.smith.segarra0 -
Google Docs Paranoia
Recently, there has been a lot of great information on SEOmoz about using Google Apps, Docs, etc. However, I suffer from Google paranoia (the fear that Google can see what's going on in my Google products). Is this fear unreasonable? Should I be concerned about using Google Apps, docs, analytics, etc. for SEO data (including keyword position tracking, back link analysis, etc)?
Reporting & Analytics | | Gyi0