Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google results not close to matching MOZ PA and SA - Why?
I'm trying to convince a boss and another client/partner of the value of MOZ, but it's a challenge when Google's results and the MOZ analytics are way off. For example, when I do non-personalized (MOZ toolbar) Google search for "stair lift kansas city" I see some results that make sense. Further down the page I see my prospective client, kclift.com -- it's a new site with poor MOZ grades and almost no backlinks. It has a PA of 1 and a DA of 4. Yet it ranks above a YP site that has a DA of 89, and another site that has a PA of 27 and a DA of 14. What gives? When I used an in-private Bing search for those terms this site didn't display in the 1st 4 pages. That makes sense with these MOZ rankings and the new site. So how does the site get page 1 results on Google for the same terms? (I should note that I've gone to this site a lot, for the client, and done other related searches. I live in Charlotte, which is why I've included Kansas City in the search terms. With this search, I've tried several ways to do a Google search that is truly anonymous, but I'm not sure I've succeeded. I've erased browsing data, used incognito, used other browsers, and used the MOZ non-personalized Google search. All from the same IP though. Does Google track that? If so, how can one do a truly anonymous search on Google?) All suggestions welcome. Thanks! Richard S.
Moz Pro | | RFS550 -
How do I know which keywords to track in my MOZ Campaigns?
Good morning Mozzers from the London, England. This may be a silly question to many of the community, but here goes "How do I know which keywords to track in my MOZ campaigns?" I work in the gyms section in the UK and am tracking keywords like "Gyms in Liverpool" or "Manchester gyms" which send me some traffic. I can see search terms like the above do send me traffic from Google's Webmaster Tools, but obviously this tool won't highlight the keyword opportunities I am missing out on because I currently don't rank. Any guidance on this dilemma would be hugely appreciated, Kindest regards Ben
Moz Pro | | Bendall0 -
Moz Local
Hey Does anyone know when Moz Local will be available in the UK? Please share your experience. I'd like to know how it's actually helped your business and client base. Thanks so much Gary
Moz Pro | | GaryVictory0 -
Do I need a new moz campaign for a subdomain?
Will moz automatically track my new subdomain or do I need to set up a new campaign for it?
Moz Pro | | SamCUK0 -
Moz is incorrect
So i have had my latest crawl, and the keyword report is showing that using google.co.uk i have some keywords not within the top 50 - i have just done a spot check (coping & pasting) direct from Moz to google.co.uk and out of 4 i have checked 1 keyword i am number 4 and 1 keyword i am page 3 - therefore within the top 50. This leads me to believe that there is something wrong with Moz - here is 1 example "land in the cayman islands" i am showing in moz as not in the top 50 when i am showing on Google as number.4
Moz Pro | | OasisLandDevelopment0 -
MOZ TEAM - your system done broke my status as a guru
Dear MOZ TEAM - WHY, when I have 2990 moz points, and my profile shows guru, does my profile banner in the Q&A itself now say ASPIRANT? Such a cruel world this has become ! I know I've wanted all along to have you come up with any title other than "guru" because that word is so dreaded in our world. Yet seriously - aspirant? I feel like Roger got drunk one night and decided "let's tease Alan"...
Moz Pro | | AlanBleiweiss0 -
What did I do wrong? What did they do right?
I have only been here a little over a week so bare with me if I am out of protocol by asking this here if there is a more appropriate place for me to have done so. Anyhow, i own the domain www.lowloans.com and just a few months ago, this domain was ranked 1 on Google for the search term "low loans" and "lowloans" Now, that domain is nowhere to be found. I don't even know where to go to begin to work on this. With respect to the search lowloans, the sites that get top spot don't have a lot of info that I have been able to find that would help me better my position. I must be doing something wrong in my campaign as all of top 3 sites on google, don't rank within my campaign "rankings" report. Ugh. Where to start? Is there some section within SEOmoz that would give me the info I need to reclaim the top spot for my desired search terms? I am still making my way through this site so if I am asking something obvious to you vets, please be patient with me.
Moz Pro | | FailuretoPlan1 -
Link from Dr. Oz web site not being seen, what are we doing wrong?
Newbie so this may be SEO 101... Quick back story: My wife Lauri is a Dream Expert who has been on over 5,000 radio shows, TV shows like Good Morning America, the Today show, the View, in major magazines and newspapers, the list goes on and on, I'm not bragging actually the opposite. Lauri barely exists online and to top it off we don't seem to be getting any "link juice" from some big sites that have linked to us. Here is just one example of many: Our site: theDreamZone.com Here is a page on Dr Oz site that links to us, anchor text at top right "Click here": When you use On site Explorer this link isn't there, what are we doing wrong? Our plan is to build a new site (LauriTheDreamExpert.com) and get these links and many others pointing to us correctly so she can get the exposure she's earned. Lauri was also on the Dr Oz TV show : Watch it if you like. Thanks for any wisdom, Mike
Moz Pro | | Mike-Dream0