Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz can't see my link from an external site
Hi, I'm hoping someone can shed some light on this. I've had a followed link created from http://www.dubaisharks.com/ pointing to my website - http://www.yallarugby.com but in the reports it doesn't show that there is a link coming fro this website. The link has been active for a few months now, but still no record of it. Any ideas?
Moz Pro | | SeoSheikh0 -
Could client be skewing search results?
I'm working with a client that is searching Google numerous times per day for various phrases that his site may rank for. A lot of these he uses the same search phrase multiple times/day to see if there is any change. Unfortunately, these are not the phrases I have optimized his site for. Moz is now displaying some of these search phrases along with associated pages and grading them as F - because they are not optimized for these phrases. Is it possible that my client caused this by doing the searches? If I already have particular pages grading an A for the targeted term chosen, should I attempt to add elements to move the F to a better position for the new keyword phrase being suggested? (I always thought you should target one page/phrase. Thanks!!
Moz Pro | | JanetJ0 -
Moz says I am missing titles and meta tags and have duplicate content
I just redesigned my website and suddenly traffic has dropped. Moz says I am missing titles, meta tag descriptions and have a lot of duplicate content. My site is http://skigenie.com and is full of unique and hand written content,. Are there any wordpress plugins that will add titles etc to my pages (some of the added ones are custom)? Any help would be much appreciated!
Moz Pro | | flexy0 -
Is there a way to get moz keywords analysis % through the api ?
I'm looking to integrate moz keywords analysis grade (in %) into my keyword research spreadshit as a column. Is there a way to get the % through the API ? Thanks
Moz Pro | | Sindicic_Alexis0 -
Where are my SEO Moz resources
I logged in, changed my password and now I can't find my SEO resources. I need to pull a report quickly, and can't find what I need. Please help ASAP!
Moz Pro | | bcbsm0 -
MOZ toolbar not recognising me as signed in
Since SEOMoz became MOZ, my toolbar is no longer working. Is this a known issue? I'm definitely signed into the account (hence this email), but the toolbar still requests login. Apologies if I'm not the only one to report it. Robb
Moz Pro | | experiencedays0 -
Open Site Explorer Backlink Numbers Wrong?
So, I have a new site that we are currently building links for, lots of hard work. Anyway, I was a bit shocked when I viewed my site through Open Site Explorer and found only 4 backlinks!!!!! Alexa.com, Majestic, and Google Webmaster Tools don't show the exact same numbers, but their figures are all MUCH higher, 4 to 566 for example. What gives Site is www.powerequipmentplus.com for those who want to look themselves. Thanks for your time and concern.
Moz Pro | | DRPower0