Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How long does it takes for Moz pro to find backlinks and increase my score?
Hi, I'm running a website for tourism & travel called Visit Guide or دليل السفر and i have many backlinks from many websites ( DoFollow & NoFollow ) They are older than 10 days and Moz has not discovered them yet nor found them? How to push Moz Pro to find my backlinks? or how to SpeedIndex my backlinks in other words?
Moz Pro | | VisitDotGuide0 -
Is the Moz Spam Score a reliable detection of PBNs?
Hello Moz community I am looking at a couple of websites to guest post on and it feels almost too easy. I am concerned these sites may be part of a PBN. Their spam scores are quite low however (6% - 11%). Is the new Spam Score on Link Explorer (not Open Site Explorer) a reliable tool to vet PBNs? I asked a member of the Moz team but they didn't know. Perhaps another will. Here is the website http://www.dspacecloud.org/ Thank you in advance for your opinions.
Moz Pro | | Avid-Panda0 -
Why is MOZ crawl is returning URLs with variable results showing Missing Meta Desc? Example: http://nw-naturals.net/?page_number_0=47
Can you help me dive down into my website guts to find out why the MOZ crawl is returning URLs with variable results? And saying this is missing a description when it's not really a page? Example: http://nw-naturals.net/?page_number_0=47. I've asked MOZ but it's a web development issue so they can't help me with it. Has anyone had an issue with this on their website? Thank you!
Moz Pro | | lewisdesign0 -
A blank moz bar
Hey MOZ peeps, iv come across something strange. The website http://abundantiaentertainment.com/ displays a "blank" MOZ bar ?! Can anyone help me understand this Thank you in advance
Moz Pro | | Silverwaring0 -
Could client be skewing search results?
I'm working with a client that is searching Google numerous times per day for various phrases that his site may rank for. A lot of these he uses the same search phrase multiple times/day to see if there is any change. Unfortunately, these are not the phrases I have optimized his site for. Moz is now displaying some of these search phrases along with associated pages and grading them as F - because they are not optimized for these phrases. Is it possible that my client caused this by doing the searches? If I already have particular pages grading an A for the targeted term chosen, should I attempt to add elements to move the F to a better position for the new keyword phrase being suggested? (I always thought you should target one page/phrase. Thanks!!
Moz Pro | | JanetJ0 -
Does SEOmoz give a way to know what link on what page produces the 404 errors that SEOmoz is telling me I have??
SEOmoz gives me a report of 404 errors on my site. Do they give a way to know from what link on what page produces the error?
Moz Pro | | MeridianGroup0 -
New campain which give exactly same results as a completed old one after 2 hours
I started a campain a long time ago. 10,372 pages have been crawled and I have the results. Great. I started a new campain today (after migration of new code) and after two hours, the campain shows excatly the same results than what the the previous one completed in 4 days, as if it was the same crawl. Then the crawl stops and we see that the next one is in 7 days... Before there was a way to launch a campain after code modification to see right away the benefits. How can we do to see ifi the changes brought something without waiting for the next crawl? Thank you.
Moz Pro | | celyfred0 -
SEO Moz Tools - too many on the page links result driving me nuts
A while back I remember Rand and I having a conversation about how many links on the page and up until that point I had followed the NO MORE THAN 100 links on a page rule - which is what the MOZ tools are telling me now in the campaigns I have running. But then during a seminar both of us were holding this 100 link rule question came up and Rand commented that this was probably old hat now as the search engines can crawl a much greater number of links in the page. I was encouraged by his answer especially where ecommerce websites are concerned. But the MOZ tool is driving me nuts telling me that this 100 link rule is still something to be adhered too. It is especially frustrating when we are discussing ecommerce website sites with editable mega menus. Examples to support this question are www.bohemiadesign.co.uk or www.flowersbuydelivery.co.uk which are 2 ecommerce sites I am aware of using such mega menus that are editable and give a link count greater than 100. and I am sure there are many more sites like this, even Amazon for example. So, how much notice do we take of this warning in MOZ tools that is telling me about excessive numbers of links on the pages it lists as needing corrected?
Moz Pro | | ICTADVIS0