Handling a Huge Amount of Crawl Errors
-
HI all,
I am faced with a crawl errors issue of a huge site (>1MiO pages) for which I am doing On-page Audit.
-
404 Erorrs: >80'000
-
Soft 404 Errors: 300
-
500 Errors: 1600
All of the above reported in GWT.
Many of the error links are simply not present on the pages "linked from". I investigated a sample of pages (and their source) looking for the error links footprints and yet nothing.
What would be the right way to address this issue from SEO perspective, anyway? Clearly. I am not able to investigate the reasons since I am seeing what is generated as HTML and NOT seeing what's behind.
So my question is: Generally, what is the appropriate way of handling this?
-
Telling the client that he has to investigate that (I gave my best to at least report the errors)
-
Engaging my firm further and get a developer from my side to investigate?
Thanks in advance!!
-
-
Usually an on page audit lists all of the problems and possible reasons why they are happening, not in depth info on how to fix all the issues. That is usually the next phase, "do you want me to work on the site or do you want your dev team to track down the cause of the issues and fix them"
It also depends what type of contract you have with him of course.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best Way to Handle Multi-Language Sites
In the last year we've made a few significant changes to the structure of our site - namely adding translations for a few languages. We have historically been gaining in organic search by about 10% each month, but in the last two months we've leveled out and seen a slight dip. I am wondering if this has something to do with the addition of the second language, and namely if there's a chance we've been penalized due to duplicate content. We have almost all pages / content on the site translated by a translator, but the way the development works the site will grab the english version if a translation hasn't been added - potentially adding some duplicate content? The URL structure remains the same, other than the addiion of the language - site.com/our-tour vs site.com/de/our-tour We also haven't translated the tour name itself, so that remains the same. Just wondering if anyone has any feedback on best practices here or things I should be looking out for. Thanks in Advance.
On-Page Optimization | | mkgreyound1 -
Will shortening down the amount of text on my pages affect it's SEO performance?
My website has several pages with a lot of text that becomes pretty boring. I'm looking at shortening down the amount of copy on each page but then within the updated, shortened copy, integrating more target keywords naturally. Will shortening down the current copy have a negative effect on my SEO performance?
On-Page Optimization | | Liquid20150 -
HTTPS and HTTP both exist! How to handle?
I was asked to do some SEO work for a website and learned that just 6 weeks ago, their webmaster added an HTTPS instance of the site. Their backlinks all point to HTTP and the 6 pages that are already ranking are all on the HTTP site. I'm afraid to rock the boat by redirecting the site from HTTP to HTTPS as we may lose rank. What are some suggestions? If I just pull down the HTTPS will that hurt us? Would you just go ahead and redirect it? IF so, would you do each page individually or as a whole?
On-Page Optimization | | dk80 -
Wordpress sitemap url problem causing WMT errors
The following types of links are appearing in my webmaster tools crawl errors report under 'other'. I've noticed they are in my sitemaps ( I run wordpress and use a plugin called Google XML sitemaps). How do I get rid of this error? http://www.musicliveuk.com/bands/postname%/
On-Page Optimization | | SamCUK0 -
Duplicate Content only an Issue on a Huge Scale?
To what extent is duplicate content an issue? We have a support forum with some duplicate content because users ask the same questions. The Moz reports we receive highlights our duplicate content and page title for our support forum as a "big" issue. I'm unsure to what extent it harms our SEO, and making the support section non-crawable would impair our level of support. It would be nice to know for sure if we should be concerned about this, and if yes, how can we do it differently? Thanks, I appreciate you help. -Allan
On-Page Optimization | | Todoist0 -
Errors when checking W3C HTML after added Google Custom Search
hello, I have added google custome search to my website, and then check with W3C HTML, it report many error.
On-Page Optimization | | JohnHuynh
eg: there is no attribute "enableHistory" <gcse:searchbox-only enablehistory="true" autocompletemaxcompletions="5" au…<br="">or there is no attribute "resultsUrl" and so on ...</gcse:searchbox-only> Has anyone face with this problem, I don't know how to fix it. Please help!0 -
Why Does SEOMOZ Crawl show that i have 5,769 pages with Duplicate Content
Hello... I'm trying to do some analysis on my site (http://goo.gl/JgK1e) and SEOMOZ Crawl Diagnostics is telling me that I have 5,769 pages with duplicate content. Can someone, anyone, please help me understand: how does SEOMOZ determine if i have duplicate content Is it correct ? Are there really that many pages of duplicate content How do i fix this, if true <---- ** Most important ** Thanks in advance for any help!!
On-Page Optimization | | Prime850 -
Huge rewrite and drop in ratings
Hi, My website provider made a huge rewrite of code to make the website more SEO friendly and add more functions. It absolutely was a big step forward and gave me more possiblities to improve our site seo wise. We were actually first webshop of 200-300 on the new platform - and thats not always a good idea i guess 😉 I need some help on the best way to solve 1 big problem for us. They changed the URL structure. Ive been building the brand pages and have 2-6 linking root domains to each brand page: for the brand ej sikke lej the catalogue used to be : http://www.epleskrinet.no/ej_sikke_lej/M_16 they changed it to http://www.epleskrinet.no/manufacturers/ej-sikke-lej . I told them the same day the new version was ot the the optimal URL would be http://www.epleskrinet.no/ej-sikke-lej . It wasnt utill i saw the crawl i understood what the URL change would mean to us. We have a decent amount of organic searches and now visitors come to an empty page with an error message. to the brand ej sikke lej i have 6 linking domains going to this page. The new brand pages are a lot weaker then the old one since they dont have the ingoing links. Our site was indexed yesterday and the result rankingwise wasent pretty. How do i go about to fix this. Is it possible to -301 redirect old brad catalogues to new one ? They are not even in the new site map i am sending in. Thanks
On-Page Optimization | | danlae0