Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Need Moz SEO Wordpress Plugin With API
Re: Moz WordPress Plugin? Hi guys,
Moz Pro | | mrezair
I need some Moz SEO Wordpress Plugins For my website working with Moz API. I've already found Moz DA-PA Checker plugin Moz DA-PA Checker But Need SEO Plugins too. Any Suggestion will be appreciated.0 -
How i can increase my site Moz DA and keyword difficulty difficulty?
Is there any way to increase my site Authority through increasing DA of my site and Keyword difficulty ? my site is about gamming like https://cpsandtypingtest.com/
Moz Pro | | sathoiue80 -
Is the MOZ Community Ranking broken?
I have been busy and away from MOZ for a while, but over the years I have been an active member of the community. I have over 2000 Moz points and usually if you haven't been on in the specified time period you become unranked, however this usually updates in 24 hours of logging back in. I have been active again since Sunday and I am still unranked yet I have gained several more points answering a couple of questions. My question is - have I missed something and the system has changed (not according to the top users table) or is the system broken?
Moz Pro | | Matt-Williamson0 -
Duplicate Site Content found in Moz; Have a URL Parameter set in Google Webmaster Tools
Hey, So on our site we have a Buyer's Guide that we made. Essentially it is a pop-up with a series of questions that then recommends a product. The parameter ?openguide=true can be used on any url on our site to pull this buyer's guide up. Somehow the Moz Site Crawl reported each one of our pages as duplicate content as it added this string (?openguide=true) to each page. We already have a URL Parameter set in Google Webmaster Tools as openguide ; however, I am now worried that google might be seeing this duplicate content as well. I have checked all of the pages with duplicate title tags in the Webmaster Tools to see if that could give me an answer as to whether it is detecting duplicate content. I did not find any duplicate title tag pages that were because of the openguide parameter. I am just wondering if anyone knows:
Moz Pro | | MitchellChapman
1. a way to check if google is seeing it as duplicate content
2. make sure that the parameter is set correctly in webmaster tools
3. or a better way to prevent the crawler from thinking this is duplicate content Any help is appreciated! Thanks, Mitchell Chapman
www.kontrolfreek.com0 -
Moz shows duplicate content, but URL's are tagged with campaign tags
Crawl diagnostics shows a lot of pages with duplicate content, but when I check the details, I see that it lists the same page but the url contains a campaign tag, so it's not really another page that is serving identical content... Is there a way to remove these pages out of the Crawl Diagnostics?
Moz Pro | | jorisbrabants0 -
Google Peru rank and SEO Moz
Hello, I wanna track the google Peru rank but the tool say me that im not in the top 50 and i'm sure that I'm, how to get a real ranking?
Moz Pro | | Kuna0 -
Cant understand Seo moz analysis
Did a semoz analysis of site www.1stclassdriving.co.uk and it came up with 484 pages with duplicate titles and 484 with duplicate content. Also gave me a 531st ranking for search term "Driving Lessons" (with many smaller schools with better rankings)- so I think that I may have a problem 1-Duplicate titles - I have many area pages - one for each of the post codes that we cover (some 40 odd) Each page is titled "Driving Lessons in "xxx" where xxx is the area.
Moz Pro | | Brian_Worger
Plus each Instructor has their own page which is tiled "Driving Lessons in xxx" where xxx is the more local towns/areas covered.
Is this the use of the term Driving Lessons xxx on each page the problem? The report doesnt seem to make this clear -it simply lists each page on the site 2- Duplicate content puzzles me as the content does differ although each instructor page has probably 80% content derived from a master page. Attach screenshot - hope someone can help Many thanx Brian0