Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Need Moz SEO Wordpress Plugin With API
Re: Moz WordPress Plugin? Hi guys,
Moz Pro | | mrezair
I need some Moz SEO Wordpress Plugins For my website working with Moz API. I've already found Moz DA-PA Checker plugin Moz DA-PA Checker But Need SEO Plugins too. Any Suggestion will be appreciated.0 -
How Moz takes a page title is duplicate?
Suppose i have added suffix and prefix to each of my product (ex: i have two tittles like buy online t-shirt at abc.com & buy online poster at abc.com, so in this buy online and abc.com are suffix and prefix) so .. will it take these two page tittles as duplicates ?
Moz Pro | | vayush0 -
When is the Chrome MOZ-toolbar going to be updated? It flaws into other website designs.
Hey, The MOZ toolbar destroys the layout of a lot of sites, which makes it very annoying always switching from top to buttom, then back up, etc. from website to website. This doesn't happen here, but on really a lot of other websites.
Moz Pro | | mrblue91
This shouldn't take that much time to fix, could it? Please let me know when we could expect an upgrade, or at least let us know, that you are aware of the problem. Regards,
Jet0 -
Perplexed - Errors increasing, moz rank dropping, conflicting data from other sources. Please Help.
Hi there, I'm in a bit of a bind and could seriously do with some help. For the past month I've been working with a client to resolve errors onsite. In that period my moz rank has plummeted and my errors (dupe title & content) have increased dramatically. The correlation makes sense, the increase - not so much. Here's why, screaming frog is reporting that the majority of duplicate titles have been removed or dealt with. I've also got the assurance of the developer that the suggested changes are being implemented. Unfortunately, I'm not familiar with the CMS - umbraco - and this is the first time I'm working with this particular developer, so I'm not sure how to gauge progress without using moz tools. So, here are my immediate questions: 1. Why would I get different results from Screaming Frog and Rogerbot? 2. Has anyone here been in a similar situation and could they recommend areas to watch for errors creeping in? 3. Is there a way of identifying which errors have occurred when? Narrowing it down to the week each error occurred would be good enough. 4. Is anybody aware of any inherent SEO flaws in umbraco or common SEO mistakes made using the CMS? 5. is there anything I could provide you with to help you help me? 6. Do you have a suggestion? I'm pretty stuck. Thanks in advance.
Moz Pro | | KJDMedia0 -
Is there any report / tool that gives me last cache date for each page on my site ?
my site has several hundred pages, and it is important for me to know last crawl date of each page as well as number of pages cralwed in a particualr period ( from / to date ). is there any report in seomoz that can help for this ? or any other suggestion ?
Moz Pro | | elegantmicroweb0 -
Can I see when SEO Moz has crawled my website?
I would like to know if it's possible to see (maybe in my Google Analytics) if SEO Moz has crawled my website. I'm also curious if and where I can see when the robot of Google visited my website. Thanks!
Moz Pro | | Spotler0 -
Poor OSE results on my links
I am really disappointed with the "return on investment" I am getting from being a Pro member and using Open Site Explorer to analyze and remedy my inbound links post-penguin. My site has been crawled for over 4 months now by seomoz and OSE reprots that I have "external links" from just 25 or so domains - whereas Google webmaster tools has accurately identified well over 270 domains that link back to my site. How can do any sort of link removal strategy using open site explorer? I am paying $100/mo for SEomoz, and nothing for Google. What is the problem?
Moz Pro | | veezer0