Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Anyone out there know MOZ well enough to provide a consultation on our results?
Hi Fans of MOZ, is there anyone out there that knows MOZ like an uber user and understands the details on crawl reports and what they mean? I'm looking for someone to review my MOZ results and give me some practical suggestions to work on with my SEO and web teams. Any takers?
Moz Pro | | NCPG
Bob Preston,0 -
Why my moz domain authority dropped over night
Dear Moz team Please do let us know why my domain authority (loanonmind.com) suddenly dropped over night
Moz Pro | | experts90 -
Setting up MOZ to run on Staging
Hi Moz, We would like to setup Moz to run on our Staging Server. This would be extremely valuable as it alerts us to new SEO issues/risks in a controlled and secure environment that is not exposed to production. Our internal team has recommended potentially setting up a reverse proxy server that will validate either via Moz's/Rogerbot's http header or IP and allow Moz access to our Staging environment to crawl. Is this something that we can setup with Moz? Are there other ideas to enable Moz to crawl our Staging server?
Moz Pro | | kriskunisch0 -
404 errors High Priority Issues in Moz Pro: change to 301 or not ?
Hi there, Moz Pro is showing us 404 errors on our site as High Priority Issues. These 404 errors regard deleted product pages, which we did not 301. Should we 301 them all backwards ? We have an ecommerce site. After reading How Should You Handle Expired Content? on Moz and a few other Q&A discussions I now know we should 301 each expired url and now we do so. My concern is with what was done in the past, and what we should do about it: for the past few years we have been leaving the pages on the site, creating a big amount of outdated url's without either content nor traffic in march our IT decided to delete these url's, and ask for a webpage removal in Google Search Console: we 301 only a 40 url's and 404 the other 3500 now 6 monthts after, we still have 2500 crawl errors in the Search Console, and Moz Pro finding each week new 404 errors Our SEO consultant says we should not bother about the errors shown in the Search Console. But I am concerned about these errors not reducing, and about Moz Pro High Priority Issues: should we 301 the url's to similar categories or products?
Moz Pro | | isabelledylag0 -
Is there a tool that can rank multiple results for a single keyword?
Hi all, I have searched the web for a tool that can tell me if more than 1 landing page is ranking for a single keyword. I have noticed sometimes that a single keyword query can trigger up to 3 landing pages within the 1st SERP. I found an old bookmarklet from Lunametrics that might do the trick using Search Console data, but the link is no longer working. Any other pointers, or can Moz do this as well? Thanks!
Moz Pro | | rodelmo40 -
How many times have you reauthorized your facebook account for Moz analytics?
I have hardly any facebook data, because Moz constantly needs me to reauthorize it. It's extremely annoying, but I wanted to make sure this is a Moz error and not a user error. Is it just me or do other people have to keep doing this as well? Thanks, Ruben
Moz Pro | | KempRugeLawGroup3 -
Open Site Explorer produces better results for www.50campfires.com than 50campfires.com
Our website url is 50campfires.com and www.50campfires.com redirects to 50campfires.com Look at the Open Site Explorer results for the two and www.50campfires.com has better results. Does google have an issue with this or may it be confusing them? We have redirects in place. How would I go about fixing this? More on what strategies would improve Page Authority and Page Rank. Thanks in Advance. 79JtBKA.png lrHwsOU.png
Moz Pro | | revonick0 -
Can not start a SEO MOZ campaign?
I keep getting this message, I have a 301 from my domain to my domain with the www prefix. What's the deal? We have detected that the domain www.photosbykristopher.com and the domain photosbykristopher.com both respond to web requests and do not redirect. Having two "twin" domains that both resolve forces them to battle for SERP positions, making your SEO efforts less effective. We suggest redirecting one, then entering the other here.
Moz Pro | | KrisPhoto0