902 Error and Page Size Limit
-
Hello,
I am getting a 902 error when attempting to crawl one of my websites that was recently upgraded to a modern platform to be mobile friendly, https, etc.
After doing some research it appears this is related to the page size. On Moz's 902 error description it states: "Pages larger than 2MB will not be crawled. For best practices, keep your page sizes to be 75k or less."
It appears all pages on my site are over 2MB because Rogbot is no longer doing any crawling and not reporting issues besides the 902. This is terrible for us because we purchased MOZ to track and crawl this site specifically.
There are many articles which show the average page size on the web is well over 2MB now: http://www.wired.com/2016/04/average-webpage-now-size-original-doom/
Due to that I would imagine other users have come up against this as well and I'm wondering how they handled it. I hope Moz is planning to increase the size limit on Rogbot as it seems we are on a course towards sites becoming larger and larger.
Any insight or help is much appreciated!
-
Hi there! Tawny from Moz's Help Team here.
I'm so sorry that you're running into persistent 902 errors from your site crawl! Unfortunately, this is a technical limitation of our tools. We're in the process of building a bigger, badder, better crawler, but it isn't quite ready yet.
You're certainly not the first person to write in about this issue, so I'll be sure to take your feedback to our product team! The ability to crawl larger pages is going to be more and more important as time moves forward.
Thanks for writing in to let us know you were having issues! If you have follow-up questions or need any other help with the tools, feel free to write in to help@moz.com and we'll get things sorted out for you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Domain Analysis Limit
Hello, even though we are a Moz Pro user we have a 'part-timer' who keeps the client metrics scoreboard up to date. Rather than have him running wild in Moz Pro we bookmarked the 'Domain Analysis' page and he knows that only 3 can be reported on per day. If it was an immediate need then we would use Moz-Pro. The issue was reported to me that he is receiving 'Search Limit reached' and do a moz-pro trial. So, has the 3 per day limit been changed? Thanks, Dave
Moz Bar | | 7thPowerInc0 -
Crawl Test : Error attempting to request HTTPS page
Hallo When I launch the crawl report I get csv file with this error : 804 : HTTPS (SSL) error encountered when requesting page.
Moz Bar | | micvitale
Error attempting to request page; see title for details. Website is https://bastabollette.it0 -
Moz is finding phantom pages
I suddenly have 4xx errors in my crawl diagnostics because pages with “/%3C/div” added to the end of the URL that are linked from the normal page can't be found. I didn't create the pages, and they don't exist, but Moz thinks that they do. I went back through to see if any changes in WordPress, theme or plugins versions might be the cause, but this is the only site that I have this issue, so I don't think that is it. Does anyone have an idea what causes this?
Moz Bar | | samuelldrew0 -
MOZ On Page Grader Not Seeing Changes
Hello all, I've been doing some SEO work recently and using the really handy A-F rating on MOZ on page grader as a guide. It's worked great so far on my sites up until today. I've made some small changes to a site such as the title and MOZ isn't seeing these changes. For example if I were to put 'banana' in the title of one of my other sites and re-run the on page grader with 'banana' as a keyword, instantly MOZ would say this keyword is used in the title once. I've done the same to another site and the same thing hasn't happened. I did a test and wrote all the keywords from the title in the on page grader and it still said they appear 0 times in the title. Am I missing something here? The site is different from the others, it's flat html and not CMS, but the SEO changes I've made are still the same. Thanks!
Moz Bar | | HB170 -
Error 605
Hi I have been getting the ollowing error on my dashboard the last 6-8 weeks error 605: Page Banned by robots.txt, X-Robots-Tag HTTP Header, or Meta Robots Tag It does seem to be able to crawl the site so what does this mean? The only issue I see is that I don't get any crawl errors listed. Could this be a prob with moz? I had this site checked on a different account on moz and there didnt get the error! The site is http://www.copyfaxes.com Thanks
Moz Bar | | henya0 -
How can I find the old ERRORS and WARNINGS report in the NEW Moz design?
I'm looking for a complete list of errors and bugs that need to be fixed within a website. I used to use the MAIN tool (at least it seemed it was the most popular) but now that its just MOZ.com I can't seem to find that great report. It had data such as: 1. List of pages with Title Tags too long 2. List of pages with Description Tags too long 3. List of RED errors and YELLOW warnings, BLUE somethings... etc... Ring a bell? I LOVED this report, where can I find this data? Thanks! Derek
Moz Bar | | DerekM42420 -
On Page Grader Question
Hi there, I have just used the On Page Grader on this page: -removed- and checked the grade for the following keywords together: wedding venues bath The On Page Grader has found 15 instances on the page for this keyword. When i view the page source and ctrl-f and search for ''wedding venues bath'', it only finds 8. Is there an issue with the On Page Grader atm?
Moz Bar | | jennie.evans0 -
Crawl Diagnostics: Exlude known errors and others that have been detected by mistake? New moz analytics feature?
I'm curious if the new moz analytics will have the feature (filter) to exclude known errors from the crwal diagnostics. For example, the attached screenshot shows the URL as 404 Error, but it works fine: http://en.steag.com.br/references/owners-engineering-services-gas-treatment-ogx.php To maintain a better overview which errors can't be solved (so I just would like to mark them as "don't take this URL into account...") I will not try to fix them again next time. On the other hand I have hundreds of errors generated by forums or by the cms that I can not resolve on my own. Also these kind of crawl errors I would like to filter away and categorize like "errors to see later with a specialist". Will this come with the new moz analytics? Anyway is there a list that shows which new features will still be implemented? knPGBZA.png?1
Moz Bar | | inlinear0