Seo moz has only crawled 2 pages of my site. Ive been notified of a 403 error and need an answer as to why my pages are not being crawled?
-
SEO Moz has only crawled 2 pages of my clients site. I have noticed the following.
- A 403 error message
- screaming frog also cannot crawl the site but IIS can.
- Due to the lack of crawling ability, im getting no feed back on my on page optimization rankings or crawl diagnostics summary, so my competitive analysis and optimization is suffering
Anybody have any idea as to what needs to be done to rectify this issue as access to the coding or cms platform is out of my hands.
Thank you
-
Hi Joel,
I have indeed!
I'll take all this information to the development team and hopefully have this issue resolved asap.
Thanks once again to everybody for their input.
Regards,
Andy
-
Hey Andy,
You've gotten a lot of good info here and in your email with Maura. I just wanted to add in here that the crawl trouble you're having (at least with our system) is with the http://www.inoncology.com/ home page itself.
You've got a 301 redirect from the root to the www subdomain, the www then returned a 403 forbidden to our crawler.
It may have something to do with the verification you require, it may be something more obvious on the server level. It's tough to say.
Either way, you'll want to get this addressed to make sure your site is crawlable.
I hope that helps.
Cheers,
Joel. -
Hi Lynn
Thank you for taking a look into this issue, your feedback has given me food for thought.
I am going to forward on your feedback to the powers that be on the web development side of things and see if this is an answer to the problem.
I've just looked at http://www.inoncology.com/home/ and from first review, it seems that this is where the crawlers are breaking down, as it comes to a dead end.
Hopefully this is the reason.
Thank you once again
-
Hi Andy,
Not sure why you are seeing a 403 error (I don't see this in the headers), but you seem to have an issue with your rel=canonical tag on your homepage. It is showing a relative link to "/home" which leads you to this page: http://www.inoncology.com/home/ which is actually showing a 200 OK status, but is obviously not!
Digging into the headers a bit more it seems you might have a circular redirect going on before that, I am seeing /home redirect to /content/internet/pm/inoncology/com_EN which seems to redirect back to /home/.
No 403 in all of those, but maybe the combination of wrong rel canonical tag plus the mixed up redirects is causing the moz crawler and screaming frog to choke?
-
Not at all Jesse. Thank you for taking a look
Brief summary - the pharmaceutical industry has to have a disclaimer on their pages for legal reasons. Forcing a selection/disclaimer is required by law. If you select the first option, it'll take you to the web address intended.
In terms of it being related to the disclaimer, SEO Moz was indexing it with this disclaimer up until a few months ago. There's no robot txt file as you stated, so that also cannot be the reason for this.
hhhhmmm........the mystery continues!!!
-
no robots file so it's not that. I do notice that your landing page has an annoying popup that forces a decision and then redirects you upon choosing it... I'm not 100% sure but that might have something to do with it. I can't seem to get past that and onto the site itself honestly. Everytime I click a selection I get to another page with a popup.
If I were an average internet browser *(which in this instance I am) I would leave your site and not return. Poor user experience hurts ultimately, and that was what I experienced.Not trying to be rude, just trying to be helpful. Sorry if I came off cross.
G'luck!
-
the url is http://www.inoncology.com/
-
Please share the URL of your website so we can check.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
50,000 4xx errors listed in MOZ report :(
HI, I've just had a look at a customers MOZ report and discovered nearly 50,000 errors listed!! The site is a non dynamic, non database driven site so where these have come from is beyond me. Example - (Both links open in a new window)
Moz Pro | | Skips
LIVE PAGE - http://lunnonwaste.com/licenses/Heatherland-Limited-H&S-Policy-Document-JAN-15.pdf
NON EXISTANT PAGE - http://lunnonwaste.com/licenses/licenses/Heatherland-Limited-H&S-Policy-Document-JAN-15.pdf - if you hover over any links you'll see they're ALL ......./licenses/licenses/.......... which doesn't exist! It's the file system that seems to be the problem - .........../licenses/licenses/........... and this gets added onto (............./licenses/licenses/licenses/licenses/.............. , up to the 50,000 page errors LOL The problem is self replicating. It's very odd and not something I've ever seen before, NON of these 'extra' pages are listed on the server, so where are they coming from? Any suggestions or help would be gratefully appreciated1 -
Meta Tag Descriptions not being found in Moz Crawls
Hey guys, I have been managing a few websites and have input them into Moz for crawl reports, etc. For a while I have noticed that we were getting a gratuitous amount of errors when it came to the number of missing meta tags. It was numbering in the 200's. The sites were in place before I got here and a lot of the older posts no one had even attempted to include tags, links of the page or anything. As they are all Wordpress Sites and they all already had the Yoast/Wordpress SEO plug-in installed on them, I decided I would go through each post and media file one at a time and update their meta tags via the plug in. I personally did this so I know that I added and saved each one, however the Moz crawl reports continue to show that we are missing roughly 200 meta tags. I've seen a huge drop off in 404 errors and stuff since I went through and double checked everything on the sites, however the meta tag errors persist. Is this the case that Moz is not recognizing the tags when it crawls because I used the Yoast Plugin? Or would you say that the plugin is the issue and I should find another way to add meta tags to the pages and posts on the site? My main concern is that if Moz is having issues crawling the sites, is Google also seeing the same thing? The URLS include:
Moz Pro | | MOZ.info
sundancevacationsblog.com
sundancevacationsnews.com
sundancevacationscharities.com Any help would be appreciated!0 -
Solving 'duplicate content' for page 2 of X for 1 blog post
Hi to all SEO wizards, For my Dutch blog google-plus-marketing.nl I'm using WordPress Genesis framework 2.0 with news theme pro 2.0 responsive theme. I love the out of the box SEO friendliness and features. One of those features is that it allows for a blog post or page to be divided into several pages. This results in MOZ signaling duplicate titles for all pages after the 1st page. Now I was thinking that a canonical url set to the first page should do the trick for me as I reason that the rank will go the the first page and the rest will not be seen as duplicates. Genesis does some good stuff on it's own and places the following meta tags in the header for the first page. All looks well and my question is about the same meta tags for the 2nd page and higher that I pasted below this one for the 1st page. Meta tags page 1 of X for blog post Meta tags page 2 of X for the same blog post Would it not be better to point the canonical url for page 2 till X to always point to the first page? In this case:
Moz Pro | | DanielMulderNL0 -
How do you get your web site recrawled with Moz without waiting for a week?
My initial crawl was screwed up because of a no follow that needed to be removed. I would like Moz to recrawl the site right away so I can find any other errors.
Moz Pro | | Ron_McCabe0 -
Site Redesign Launch - How Can I crawl for immediate review
Just redesigned my site and want to have a crawl done to check for errors or any items which need to be cleaned up. Anyone know how I can do this as SEOMoz only crawls once per week. Thanks!
Moz Pro | | creativemobseo0 -
SEO moz poor rankings reporting
How are we supposed to send ranking reports to clients if the rankings are constantly reporting false metrics This is not the first time this has happened, but the rankings are WAAAAAAY off. What is the point of reporting rankings that are not accurate? What is the possible cause of this. Is there a solution? What is SEOmoz doing to improve their ranking programs. Our client actually ranked #2 for "northern Virginia HVAC contractors" when moz had us not in the top 50! How are we supposed to bank on information that is so consistently unreliable? northernvirginiahvaccontractors.png northernvirginiahvaccontractorsrank.png
Moz Pro | | imageworks-2612900 -
When is the new SEO moz Rank Tracker tool coming?
Anyone know when is the new SEO moz Rank Tracker tool coming? Getting a little flustered with the rank tracker tool, its bit annoying doing one by one. 😞
Moz Pro | | waheeed0 -
Looking for Advice on SEO for Minimilistic Site
Hi everyone, I found this site and it looks like a great place for us to start, we have just opened the doors to our new website fenwaymedia thats a dot co dot uk. Now Ihave tried to maintain the most minimilistic design throughout but I believe with this comes certain drawbacks - - - - S E O which is why i am here. My thought were that I would draw my clients from the Web Design, web Marketing & Blog page which I will starting in 3 hours be constantly updating. but Im not sure Im doing the right thing. I am trying to target Edinburgh, Scotland although we would take work from anywhere, if anyone could spare us a moment, I would greatly appreciate any help, tips or comments that might help us attain some visibility. Thanks in advance and thanks for looking. Kind regards, Craig
Moz Pro | | fenwaymedia0