Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Moz crawler not working
Hi Moz crawler keep failing on my site with the error showing : Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. I'm not sure what am I missing out.. this is my robots.txt.. i don't think Im missing anything else.. https://www.wearefutureheads.com/robots.txt can the support team help ?
Moz Pro | | teikh0 -
moz crawl is stopped?
moz stopped indexing the links due to some updates? can some one confirm me thanks
Moz Pro | | 42409300125323700 -
Moz Domain Authority 27
Hi there, I have been reading up about Domain Authority being important to rank though I have a question about it. When trying a few google searches and using a tool to see the Moz Domain Authority i notice the following. I won't mention sites or keywords, happy to send them via private message if required. A site ranks 1st position for most keywords including the main keyword that related to their industry, a highly competitive keyword. Their Moz Domain Authority 27, Est. Visits: 265,000, Pinterest shares 72, Facebook shares 191 and unique domains that point to it are 55. Yet when I look at another site, optimising for the same keyword, Moz Domain Authority 50, Est. Visits: 60,000, Pinterest shares 35000, Facebook shares 20000 and unique domains that point to it are 521. I would have thought that the second site ranks 1st for the keyword and the first one mentioned which is in 1st position I would have expected to be on page two or three. as I scroll down the first page for the term, all sites except for one have higher DA, visits, shares and so on. If a high PA is required can someone help me to understand the above example? Whn a low PA site ranks much better for a primary keyword than all other sites that have a higher PA than it?
Moz Pro | | IvanaDaulay1 -
I have a Forum, so as a result I have a lot of links Is that a problem?
I'm looking at my initial reports, and it says one of my problems is that I have too many links. should I be concerned? Because part of my site is a forum.
Moz Pro | | taychatha0 -
Local search results reporting
Is there a way I can report on local search results for our keywords by city?
Moz Pro | | RPM5
For example could I specify Phoenix or Chicago and then see how we rank as if we did the search in that city?0 -
My Moz domain authority fell last week but so did all 3 of my competitors. What could cause that?
As a loyal Moz Pro subscriber I track my site's authority, trust and links against 3 similar competitors. Last week my authority fell from 61 to 60 but all 3 competitors saw a drop in their authority that week too. Was the Moz domain authority calculation changed? Did anyone else see a drop or has something odd happened to just our market? Any ideas? Not sure if this is something I should address or just shrug and ignore. Cheers
Moz Pro | | SteveBrumpton0 -
My rankings report is giving out wrong information
Hi, i am very puzzled about the rankings report. For some reason the past couple of weeks it is giving wrong information for example, my site www.in2town.co.uk the report says that the keyword gastric band hypnosis is eight in google on page one but when i have checked i cannot find it. It also says the word gastric band hypnotherapy is sixth place in google on page one can anyone please let me know what is going on
Moz Pro | | ClaireH-1848860 -
Open Site Explorer results vs. Google Webmaster Tools results
I've been comparing the links to my domain that OSE and GWT show and GWT shows many more links than OSE. Can anyone explain the difference? Does Google report no follow links that OSE does not?
Moz Pro | | cartersnipes0