Seo moz has only crawled 2 pages of my site. Ive been notified of a 403 error and need an answer as to why my pages are not being crawled?
-
SEO Moz has only crawled 2 pages of my clients site. I have noticed the following.
- A 403 error message
- screaming frog also cannot crawl the site but IIS can.
- Due to the lack of crawling ability, im getting no feed back on my on page optimization rankings or crawl diagnostics summary, so my competitive analysis and optimization is suffering
Anybody have any idea as to what needs to be done to rectify this issue as access to the coding or cms platform is out of my hands.
Thank you
-
Hi Joel,
I have indeed!
I'll take all this information to the development team and hopefully have this issue resolved asap.
Thanks once again to everybody for their input.
Regards,
Andy
-
Hey Andy,
You've gotten a lot of good info here and in your email with Maura. I just wanted to add in here that the crawl trouble you're having (at least with our system) is with the http://www.inoncology.com/ home page itself.
You've got a 301 redirect from the root to the www subdomain, the www then returned a 403 forbidden to our crawler.
It may have something to do with the verification you require, it may be something more obvious on the server level. It's tough to say.
Either way, you'll want to get this addressed to make sure your site is crawlable.
I hope that helps.
Cheers,
Joel. -
Hi Lynn
Thank you for taking a look into this issue, your feedback has given me food for thought.
I am going to forward on your feedback to the powers that be on the web development side of things and see if this is an answer to the problem.
I've just looked at http://www.inoncology.com/home/ and from first review, it seems that this is where the crawlers are breaking down, as it comes to a dead end.
Hopefully this is the reason.
Thank you once again
-
Hi Andy,
Not sure why you are seeing a 403 error (I don't see this in the headers), but you seem to have an issue with your rel=canonical tag on your homepage. It is showing a relative link to "/home" which leads you to this page: http://www.inoncology.com/home/ which is actually showing a 200 OK status, but is obviously not!
Digging into the headers a bit more it seems you might have a circular redirect going on before that, I am seeing /home redirect to /content/internet/pm/inoncology/com_EN which seems to redirect back to /home/.
No 403 in all of those, but maybe the combination of wrong rel canonical tag plus the mixed up redirects is causing the moz crawler and screaming frog to choke?
-
Not at all Jesse. Thank you for taking a look
Brief summary - the pharmaceutical industry has to have a disclaimer on their pages for legal reasons. Forcing a selection/disclaimer is required by law. If you select the first option, it'll take you to the web address intended.
In terms of it being related to the disclaimer, SEO Moz was indexing it with this disclaimer up until a few months ago. There's no robot txt file as you stated, so that also cannot be the reason for this.
hhhhmmm........the mystery continues!!!
-
no robots file so it's not that. I do notice that your landing page has an annoying popup that forces a decision and then redirects you upon choosing it... I'm not 100% sure but that might have something to do with it. I can't seem to get past that and onto the site itself honestly. Everytime I click a selection I get to another page with a popup.
If I were an average internet browser *(which in this instance I am) I would leave your site and not return. Poor user experience hurts ultimately, and that was what I experienced.Not trying to be rude, just trying to be helpful. Sorry if I came off cross.
G'luck!
-
the url is http://www.inoncology.com/
-
Please share the URL of your website so we can check.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How i can increase my site Moz DA and keyword difficulty difficulty?
Is there any way to increase my site Authority through increasing DA of my site and Keyword difficulty ? my site is about gamming like https://cpsandtypingtest.com/
Moz Pro | | sathoiue80 -
Crawler errors or Page Load Time? What affect more to SEO
Hello, I have a page with a forum and at this moment the moz report says that have 15.1k of issues like url too long, meta noindex, title too long etc. But this page have a load time realy sloooow with 11 seconds. I know i need fix all that errors (i'm working on this) but... What is more important for SEO? The page load or that type of error like duplicate titles etc. Thank you!
Moz Pro | | DanielExposito1 -
What SEO tools do you use in conjunction with Moz?
It seems like most people use multiple SEO tools. I am interested in hearing what you use in conjunction with Moz and why. -Stephen
Moz Pro | | martechwiz2 -
ERROR: Too Many on-page links!
User wise, my product menu @LEDSupply.com is user friendly, but I'm concerned that it might be seen by crawlers as bad because of the TOO MANY ON-PAGE LINKS error I am getting in my moz crawl report. Is it really counting all the links in every drop-down menu? If so, is there are resource on how to fix it????
Moz Pro | | saultienut0 -
Moz Pro vs. Moz Analytics
Beta version has been out for awhile. Curious to know what people think of Moz Analytics. Personally, I'm having a hard time making the switch from Pro to Moz Analytics because I'm so used to the navigation and layout of Pro. What does everyone else think of the new Moz Analytics? Advantages? Disadvantages?
Moz Pro | | EEE31 -
Why is Moz Reporting as Duplicate Page Titles?
Our most recent MOZ crawl campaign is reporting 931 duplicate page title errors, most of which are "Product Review" pages like the following. Although there is only one review on this page, http://www.audiobooksonline.com/Cell_Stephen_King_unabridged_compact_discs.html, MOZ is reporting 15 duplicate page title, four of which I present below. http://www.audiobooksonline.com/reviews/review.php/full/0743554337/0/name/desc
Moz Pro | | lbohen
http://www.audiobooksonline.com/reviews/review.php/full/0743554337/0/rating/asc
http://www.audiobooksonline.com/reviews/review.php/full/0743554337/0/rating/desc
http://www.audiobooksonline.com/reviews/review.php/full/0743554337/0/state/asc Why is MOZ reporting these "pages" as duplicate page title errors? Are these errors hurting our SEO? How to fix?0 -
Bad code on Learn Seo Redirection info Page
Is it just me, or is the Redirection resource page missing the exclamation point (!) in this code. If so, this could really mess someone's site up if they copy and paste. http://www.seomoz.org/learn-seo/redirection http://screencast.com/t/n7lknZ32G9xF Redirecting Canonical Hostnames: The original developers at SEOmoz needed to redirect any requests that do not start with www.seomoz.org to make sure they included the www. They did this not only because it looks better, but to avoid common canonicalization errors. Redirect: http://seomoz.org/To: http://www.seomoz.org/ Redirect: http://mail.seomoz.org/To: http://www.seomoz.org Redirect: http://seomoz.org/somefile.phpTo: http://www.seomoz.org/somefile.php Solution: Add the following directive: RewriteCond %{HTTP_HOST} ^seomoz.org [NC]RewriteRule (.*) http://www.seomoz.org/$1 [L,R=301] Explanation: This directive tells apache to examine the host the visitor is accessing (in this case: seomoz.org), and if it does not equal www.seomoz.org redirect to www.seomoz.org. The exclamation point (!) in front of www.seomoz.org negates the comparison, saying “if the host IS NOT www.seomoz.org, then perform RewriteRule.” In our case RewriteRule redirects them to www.seomoz.org while preserving the exact file they were accessing in a back-reference. *emphasis added by me
Moz Pro | | squareplug0 -
How long would a SEOMoz crawl usually take for a site with around 4000 pages?
We are working through optimising a site for one of our clients and the SEOMoz crawl progress says it has been running since the 8th Feburary. It's now almost a week later and it still hasn't finished. The first run took a few days, is there any way of restarting the process?
Moz Pro | | TJSSEO0