How to detect where Google gets indexed URL's
-
Google index some kind of way some links that create duplicate content. We doesn't understand how these are created so we would like detect where Google robots find these links.
We tried:
- Moz Crawl Diagnostics but it shows 0 as Internal Link Count for these kind of links.
- Find some information from Google Analytics, that maybe there is trace (site content - all content) from visitors side. There wan't.
- We tried to find some information in Webmaster Tools under Internal link and HTML Improvements but didn't find any trace.
- Tried some search commands. Is there maybe some good one to search.
- TO search URL's form code with https://search.nerdydata.com.
-
It really isn't possible for an outsider to know why your website is generating those URLs in error; you would have to talk to your developer about that.
As far as canonicals, if your problem is page.com is getting duplicated by added parameters: page.com/?id=1, page.com/?id=2, page.com/?id=3, etc. as long as you have the canonical on page.com, all of the parameter pages will have the correct canonical on them as well. (But you are right, you should track down the source; your developer will know.)
-
Thanks you for your answer but yes I know that these are generated by our site. But problem is that I can use canonical tag for these that are indexed right now but later new ones will be created someway. Problem root isn't that we doesn't know how to use canonical, it's how to get to know where these URL's are find/indexed/detected by Google.
These kind of URL's have been there for months so we can't just hope that somehow these will be droped. We need to find some kind of solution and detect real problem.
-
If you found those URLs by doing a site: search, then those parameters are being generated by your site. (I am surprised that Google is even indexing them; I assume that pretty soon all but one will be dropped.) Here is an article that explains more about those types of duplicate pages: http://moz.com/blog/which-page-is-canonical
You can fix this by using a canonical tag on your homepage with the version that doesn't have the parameter.
-
Our front page has almost 50 duplicate versions. These are shown when we do site:oursite.com, there are /et?id=xx, /et?productId=xx, etc. In URL xx are different numbers.
-
Where are you seeing these duplicate content links? Does Webmaster Tools say that they are duplicate content? Or does this show up in your Moz crawl? What do these URLs look like?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Referral issue in Google analytics
We have an eCommerce website that counts paypal as a referral source in Analytics. The site takes people to Paypal to make a payment and then back to the website to a Thank You page once that payment has been made. Due to this, Analytics sees this as a conversion that has come from Paypal, and also records it as a referral source, when we know this is not really the case. This also distorts the data in analytics and prohibits us from clearly seeing which channels sales have come from. Is there anyway in Analytics to include Paypal as a part of the website so that it does not record Paypal as a separate referral website?
Reporting & Analytics | | Gavo0 -
Google Analytics - Average Position
Hi Just trying to get some clarity on Google Analytics Average Positions in "Aquistions/Search Engine Optimisation". For a very competitive keyword Google Analytics is saying i am on average position of 6. Is this Page 6? I am assuming position six would be 1.6?
Reporting & Analytics | | Cocoonfxmedia0 -
Google Analytics Not Working
I added the code before tag but still google not showing it is installed. Status: Tracking Not Installed Last checked: Mar 15, 2013 10:38:10 PM PDT Can someone check my domain - www.plugnbuy.com
Reporting & Analytics | | chandubaba0 -
Google Webmaster Tools not displaying all backlinks
I was asked to take a look at a site that a friend redesigned and immediately plummeted in rankings. It was a flash site where the domain redirected to a flash landing page. When the site was redesigned, they removed the flash page and put in the new site on the domain URL without changing the redirect or putting a 301 in place on the old flash landing page URL. This site didn't have many backlinks to begin with, but it did ok for some important search phrases. Now when I look in Webmaster Tools, there are only 2 backlinks showing. Meanwhile I know there are more backlinks in existence, primarily from yellowpages and citysearch. I've had them put a 301 on the old flash landing page pointing at the domain URL and they've added a canonical URL. What have I missed here?
Reporting & Analytics | | BostonWright0 -
Google News traffic spike mystery; referring URLs all blank, Omniture tags didn't fire.
Our content is occasionally featured in Google News. We recently have had two episodes where this happened, but (a) nearly all the referring URLs were blank, and (b) our backend logs show 3-4x more requests for the article in question than Omniture does. In other words, hundreds of thousands of visitors requested a URL from our site (as proven by the traffic logs), but don't seem to have come from Google News (because HTTP_REFERER was blank), and didn't execute the onpage javascript tag to notify Omniture of the pageview. Perhaps this has nothing to do with Google News, but it is too strong a coincidence that the two times we were on there recently, the same thing happened: big backend traffic spike that is not seen by Omniture. It is as if Google News causes browsers to pre-fetch our article without executing the javascript on the page. And without sending a referring URL. Has anyone else seen anything like this before? Stats from the recent episode:
Reporting & Analytics | | mcglynn
- 835,000 HTTP requests for the article URL (logged by our servers) - these requests came from 280,000 distinct IP addresses (70% US) - the #1 referring URL is blank. This accounts for 99.4% of requests. Which, in itself, is hard to believe. These people had to come from somewhere. I believe browsers don't pass HTTP_REFERER when you click from an SSL page to a non-SSL page, but I think Google News doesn't bounce users to SSL by default.That said, we do see other content pages with 70-90% blank referring URLs. Rarely 99+% though.0 -
Cookie tracking in Google Analytics
Hi How do I remove the "/?__utma=...." at the end of my URLs?We have a site http://www.jetonline.co.za/, if you click on one of the menu navigation links i.e. "fashion". A long "/?__utma=...." url appears. I understand this is for tracking as we have separate domains for each page but is there a way to remove this dynamic url and keep it hidden from users?Thanks in advance
Reporting & Analytics | | NeilPursey0 -
Why do I have few different index URL addresses?
Yes I know, sorry guys but I also have a problem with duplicate pages. It shows that almost every page of my site has a duplicate content issue and looking at my folders in the server, I don't see all these pages... This is a static Website with no shopping cart or anything fancy. The first on the list is my [index] page and this is giving me a hint about some sort of bad settings on my end with the SEOMOZ crawler??? Please advice and thank you! index-variations.jpg
Reporting & Analytics | | cssyes0 -
Organic search on google
Hi there, pl take a look at this link, there is a section which says shared results and has a star against two agencies and once article. can pl someone let me know what those starts are and how to get them? Thank you 🙂 so sorry - the link is http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=digital+mareketing+agency+los+angeles Vijay
Reporting & Analytics | | vijayvasu0