URLS appearing twice in Moz crawl
-
I have asked this question before and got a Moz response to which i replied but no reply after that.
Hi,
We have noticed in our moz crawl that urls are appearing twice so urls like this - http://www.recyclingbins.co.uk/about/ www.recyclingbins.co.uk/about/
Thought it may be possible rel=canonical issue as can find URL's but no linking URL's to the pages.
Does anyone have any ideas?
Thank you
Jon
I did the crawl test and they were not there
-
Hey Jon!
There are only two ways for these URLs to be picked. Relative links in the source code or malformed sitemaps if the <loc>tags are not on the same line as the URL</loc>
Incorrect:
<loc>www.domain.com</loc>
correct:
<loc>www.domain.com</loc>
If the linking page or referral URL is the sitemap, look in the source for the tags. If they are from another linking page, look for hyperlinks missing http://
-
Hi Jon,
Just to clarify is your report showing the domains with http and without, or with www. and without as the duplicate content?
i have seen this issue on a couple of my sites where it is reporting with and without www. and have just implemented the rel canonical to point towards one or the other as i am very confident that this will resolve the issue.
I'll let you know the outcome.
Regards,
Ben
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz bot has trouble crawling Angular JS - I believe it's seeing the SPA (Single Page Application) before Universal. Anyone else have this issue? What is the fix?
The Moz bot user agent detection settings are able to read Universal, but the Single Page Application (SPA) version partially loads on the website before Universal. Because of this, Moz (and possibly search engines) think we have massive duplicate content issues. For example, the crawl report said a particular product page (which has about 1,000 words) has 33,000 words and has duplicate content with over 300 other pages. This makes me believe it's only picking up the SPA version. Has anyone come across this, and what would be the fix?
Moz Bar | | laurengdicenso1 -
Is MOZ any good to analyze an e-commerce site? How come that a cms page can be seen as duplicate content with a category page?
Hi Guys, I've been using Moz for quite a long time now for 2 of my shops. Now I am in the process of launching the second shop and I just don't understand how is it possible that a cms static page (About US) to be seen as a duplicate content with other 96 pages - including product pages and other totally different pages such as delivery information, category pages, returns and so on. Really MOZ?? Is it me or you?? Your help would be much appreciated! Thank you!
Moz Bar | | Sorin_T0 -
On Page Grader - URL not accessible
We have tried to use the On Page Grader today and it is coming back with URL not accessible for all pages on our website. We previously used the On Page Grader on Friday 10th Nov for a couple of product pages with no issues. Since then, the only changes we have made on the websites is updating some downloadable documents. We have done this several times before and it has never affected Moz. We have not changed the page URLs, and therefore do not know why it is now not working. The pages are working fine on the website with no issues. A link to one of the pages is below. http://www.processinstruments.co.uk/products/dissolved-oxygen-monitor/ Any help would be greatly appreciated.
Moz Bar | | PiMike0 -
How can I be sure Moz Campaign for blog is working?
I set up a Moz Campaign to track my blog domain (made a separate one for our website) and added in keywords I've optimized our blog posts for. I've spent about 2 months optimizing pages and there's been no change in the numbers I see in the dashboard (specifically Search Visibility and Top National Keywords) and Rankings module. It all basically says either 0 or '--'. I think I've set this campaign up right because the website campaign has been changing and showing different results in the same amount of time. I'm going to ask the stupid question -- is this because I've set up the blog campaign wrong, my site is blocking Moz from accessing blog data or my SEO efforts simply aren't working? Please help. I need a sanity check here. Thank so much in advance!
Moz Bar | | Visier0 -
MOZ crawler 404 errors on wordpress
Hi all, I've got hundreds of issues coming up on the MOZ crawler with 404 errors, I don't know what these URL's are. Here's a couple of examples; http://www.theswagbagco.co.uk/category/watford/http%3A%2F%2Fwww.theswagbagco.co.uk%2F2015%2F10%2F15%2Fnew-products-2%2F
Moz Bar | | vaineh
http://www.theswagbagco.co.uk/2015/10/01/thank-you-epsom/http%3A%2F%2Fwww.theswagbagco.co.uk%2F2015%2F10%2F01%2Fthank-you-epsom%2F See the first one is one page with a different url appended, the second is the same thank-you-epsom url. How would I find out where these are even being linked from?0 -
Why isn't the Moz bar data populating for Yahoo sites?
The Moz bar isn't populating information for Yahoo homepage or it's verticals (i.e. homes, autos, finance, etc.), but I can get this data for other portals like AOL or MSN. I'm specifically looking for PA, mR, and DA information, but instead I get a generic "Search Profile" bar with no page/site-specific data.
Moz Bar | | AllieBell
Is there a reason Open Site Explorer data isn't populating for this particular portal?0 -
Moz Crawl Test says pages have no internal links
Greetings, I am working on a website, https://www.nasscoinc.com, and ran a Moz Crawl Test on it. According to the crawl test, only 2 of the website's hundreds of pages are receiving internal links. When I run a similar test on the site using Screaming Frog, I see that most of the pages have at least one internal link. I'm wondering if anyone has seen this before with the crawl test; and there is a way to get the crawl test to see the internal links? Thanks!
Moz Bar | | TopFloor0 -
On Page Grader can't access my URLs
HI- I am trying to grade some specific pages for keywords with the on page grader but it keeps telling me "Sorry, but that URL is inaccessible. " I can reach them via the browser and they are not https. Any thoughts? Here is a sample: www.bulkcandystore.com/kosher-candy Any help is appreciated. Ken
Moz Bar | | CandymanKen0