My page cant be crawled by MOZ
-
Hi everyone,
I have a page named zita.vn. Im using Moz to do SEO for the page, but when I use On-page grader in MOZ, I got an error that the page cant be accessed. Additionally, I got an issue with page crawling.
Google search still crawl my page normally.
Please help.
Thanks a lot.
-
Hi there,
I have been checking my site for a week, but I havent seen any new updates about my site in backlink. Im sure that since last week there are a few sites pointing a link to my website but I havent found it on Moz.
Please explain it for me.
Tks a lot,
-
The useragent is: rogerbot
-
Hi Kristina,
Thank you for your answer.
Currently to protect our site from being hacked we need your user agent to add it into our whitelisted, so you will be able to crawl our website.
Regards,
Phuong.
-
Hi there! Kristina from Moz's Help Team here.
When our crawler attempts to access your site, we are running into this error: http://screencast.com/t/CmprKKb7a "403 Forbidden".
This normally means that you're blocking us on a server level. When I ran your site in a header checker I was able to see a 200, but when I curled your site from our servers it returned a 403.
The bummer here is that I can't do anything on the support side to fix this.
This 403 error is coming directly from your server, so at this point the best thing to do will be to send this information to your web dev. They'll need to make sure our crawler (rogerbot) is whitelisted, so our tools can start working hard for you.
As always, you can contact our team on issues like this in the future by emailing help@moz.com, but hopefully this helps point you in the right direction towards a solution here.
Thank you,
-Kristina -
I'm seeing a 403 response for your website:
wget zita.vn
--2016-08-02 11:46:09-- http://zita.vn/
Resolving zita.vn... 107.167.188.173
Connecting to zita.vn|107.167.188.173|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.zita.vn/ [following]
--2016-08-02 11:46:09-- https://www.zita.vn/
Resolving www.zita.vn... 107.167.188.173
Connecting to www.zita.vn|107.167.188.173|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-08-02 11:46:10 ERROR 403: Forbidden.If Moz is seeing a 403 they're not going to be able to crawl.
-
there is a possiblity of this issue is due to TLD, as i believe Moz crawler do not support to all TLDs as of yet. this could be the only reason comes to my mind. However, i reckon to raise a suppert ticket for more certian answer for this.
-
Maybe you've set a number of threads too high and pause between requests too low? Some hosts will block IPs that send too many requests. How much is too many depends on your host.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why page load time is different in google webmaster vs what is displayed in moz?
When I analyze the site through Moz tool and compare the results with google webmaster, I am not able to figure what why Moz does not report the slow pages. Fro example this page has an avrage LCP of 3.0 sec https://www.collegehippo.com/graduate-school/programs/gre-score-business-analytics-data-analytics When I see the report in moz, it does not point to any such issue. Should I be worried about what google reports and try to fix the page?
Moz Bar | | etattva0 -
MOZ Staff: Timeline for supporting SNI?
We have moved our blog to Amazon Web Services, and our website is soon to follow. For better or worse, AWS uses SNI, which MOZ doesn't currently support. Here are some recent forum posts about it: https://moz.com/community/q/804-server-error-crawling-https https://moz.com/community/q/804-https-ssl-error This makes MOZ much, much less useful to me. MOZ staff, you have a timeline for when you'll implement support for SNI?
Moz Bar | | Atomic-Object2 -
What does the Bold/ Strong mean in Moz bar?
Under On-Page Elements in the Moz bar there is a Tag/ Location called Bold/ Strong. What does that mean?
Moz Bar | | TiffanyatElite0 -
Puzzled. High domain authority, but low page authority?
I have a question regarding how a page's authority can be low but the domain authority be high. I'm using the SERP report in moz to compare competing pages. The page I'm working on is an ecommerce category page. Moz says it has very low page authority, has been indexed for a little over a year, it's domain authority is a moderate 50, and has no root domains linking to it. It is more relevant than the other pages (has more products for sale that fit the keyword searched for), has a decent amount of copy on the page, but ranks around #10 in the search results. The competing pages are also ecommerce category pages. The 9 ranking higher have domain authority varying from 30-75, and 1 or 2 root domain links to them. Their number of products that match the keyword searched are all over the board, some have many matches, some have only 6 or 8 results. Some have a moderate amount of text on the page, some have very little text on the page. Yet they have page authority in the 30s-40s. What goes into page authority that I'm missing which would make these other pages have so much more authority when they are just slightly better in the other metrics listed above? What am I missing here?
Moz Bar | | K-WINTER0 -
Duplicate Page Titles detected, no relevant links shown
Moz is reporting duplicate page titles, but the relevant pages aren't being shown. The downloaded report doesn't show them either. Screencast: http://screencast.com/t/gkyRds6u
Moz Bar | | ElykInnovation0 -
404 Crawl Diagnostics Report MOZ
Hi, I keep getting 404's appear in the Crawl Diagnostic error warnings. How do I find out which pages are linking to these 404 pages? How is MOZ finding them? thanks Ben
Moz Bar | | bjs20100 -
Is Moz Analytics still available to all MozCon attendees?
I thought all Mozcon attendees were going to get access to Moz Analytics even after the convention was over. I can't seem to find were to access it from my campaigns manager dashboard and the Analytics landing page says im on queue to get access.
Moz Bar | | AmberHanson0 -
Not getting foreign characters in crawl diagnostics .csv
The crawl diagnostics .csv file is showing high-ascii characters instead of the correct language (foreign language website) e.g. Vietnamese, Chinese (both kinds), etc. Is there a way to get this right?
Moz Bar | | trainSEM0