How to find 20 hidden 404s
-
Hello,
We have like twenty 404s left to find. How do you find these when:
1. They don't show up in Google Webmaster Tools
2. They don't have any other internal or external pages linking to them.
3. They don't show up in site:domain.com (We have 9000 pages and only 600 show up - I fixed those out of the 600).
4. They are probably causing high bounce rates.
5. They're not in the sitemap
Thanks!
-
You should be able to crawl the entire site without increasing the RAM, once you buy the paid version.
-
Right in the paid version wont stop! You can see in their website that the limit of urls will be remove once you buy the license. http://www.screamingfrog.co.uk/seo-spider/licence/
It should catch everything, maybe you can contact their support just to make sure, they are very good on support over twitter.
-
Yes, but if it only check 397 in the free version is it going to stop there in the paid version as well. Just making sure.
Also, will it catch everything?
-
Yes I just run a craw for 500 000 urls
-
The free version checked 397 URLs. Will the purchased version check all 9000?
-
You can try crawl the site using screamingfrog, make sure you check the box “Check Links Outside Folder” in the spider menu.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Webmaster tools -Fixing over 20,000+ crawl errors
Hi, I'm trying to gather all the 404 crawl errors on my website after a recent hacking that I've been trying to rectify and clean up. Webmaster tools states that I have over 20 000+ crawl errors. I can only download a sample of 1000 errors. Is there any way to get the full list instead of correcting 1000 errors, marking them as fixed and waiting for the next batch of 1000 errors to be listed in Webmaster tools? The current method is quite timely and I want to take care of all errors in one shot instead of over a course of a month.
Intermediate & Advanced SEO | | FPK0 -
URL Parameter Setting Recommendation - Webmaster Tools, Breadcrumbs & 404s
Hi All, We use a parameter called "breadCrumb" to drive the breadcrumbs on our ecommerce product pages that are categorized in multiple places. For example, our "Blue Widget" product may have the following URLs: http://www.oursite.com/item3332/blue-widget
Intermediate & Advanced SEO | | Doug_G
http://www.oursite.com/item3332/blue-widget_?breadCrumb=BrandTree_
http://www.oursite.com/item3332/blue-widget_?breadCrumb=CategoryTree1_
http://www.oursite.com/item3332/blue-widget_?breadCrumb=CategoryTree2_ We use a canonical tag pointing back to the base product URL. The parameter only changes the breadcrumbs. Which of the following, if any, settings would you recommend for such a parameter in GWT: Does this parameter change page content seen by the user? Options: Yes/No
How does this parameter affect page content? Options: Narrows/Specifies/Other Currently, google decided to automatically assign the parameter as "Yes/Other/Let Googlebot Decide" without notifying us. We noticed a drop in rankings around the suspected time of the assignment. Lastly, we have a consistent flow of products that are discontinued that we 404. As a result of the breadcrumb parameter, our 404s increase significantly (one for each path). Would 800 404 crawl errors out of 18k products cause a penalty on a young site? We got an "Increase in '404' pages' email from GWT, shortly after our rankings seemed to drop. Thank you for any advice or suggestions! Doug0 -
Soft 404s for unpublished & 301'd content
Hi, One site I work with unpublished a lot of thin content. Great idea, right? These unpublished pages were then 301'd up to the main category page that they previously existed in. Now Google Webmaster Tools calls them out as soft 404 errors. This seems unexpected since the pages were 301'd. Here is my question; Is this a serious problem that may affect the site's overall organic results and if so what should I do about it? Thanks... Darcy
Intermediate & Advanced SEO | | 945010 -
Launching 20 new URLs to branch out from a multi brand website. How do I make sure the rankings stay?
I have a client with a multiple brand website currently. The site has landing pages and directories for each of its 20 brands under one URL. The plan is to build 20 separate brand websites with new domains and have the old content redirect to its corresponding new URL. The problem that I envision is being able to successfully carry all of the existing indexed content over while only using 301 redirects and submitting the new domains to Webmaster tools. Has anyone done this in the past successfully and do you have any recommendations?
Intermediate & Advanced SEO | | Empower_MediaMarketing0 -
My warning report says I have too many on page links - 517! I can't find 50% of them but my q is about no follow
if we put 'no follow' on some of these links does that mean the search engines won't index the no follow pages even if those pages are linked to from elsewhere? no link juice will flow from the page with the (no follow) links on? Just trying to understand why my rankings have dropped so dramatically in the last 6 weeks or so since we redesigned the site, and it might be that now we have too many links on the homepage. This is the page http://www.suffolktouristguide.com/ All suggestions appreciated!
Intermediate & Advanced SEO | | SarahinSuffolk0 -
How do you find a truely knowledgable SEO person to analyze are large site?
We are a large site, 5600 pages with local pages in almost every city across the US. We are struggling with page rank on some pages and I dont think its as simple as backlinks and its definitely not poor on-page SEO. I think we might have some truly technical issues that is causing us to get penalized in SERP's. Any agencies which analyze sites? This is NOT a job posting so please don't send me messages...I truly want to know how/where to find a solution to our problem. Thanks
Intermediate & Advanced SEO | | CTSupp0 -
Cant find my home page to seo it....aajhhhhhh
Hi Guys, This might be more of a joomla thiing than a SEO thing but it is correlated as I need to seo this pgage and i cant find it. Please help if you can, while my developer is on hols, this is driving me nuts!! I can find the article sections in Joomla 2.5 to edit all the text in my other pages but for some reason cannot find the text for the home page!!??? any ideas? Please...?? He set a lot of it up using CSS and Jquery / php etc....so im a little confused as to why I can find the html to edit.......aaahhhhhhhh Thanks guys, Im sure its quite easy!! Thanks in advance. Craig
Intermediate & Advanced SEO | | craigyboy0 -
Use of <h2class="hidden">- SEO implications</h2class="hidden">
I'm just looking at a website with <h2class="hidden">Main Navigation and <h2class="hidden">Footer inserted on each page, and am wondering about the SEO implications.
Intermediate & Advanced SEO | | McTaggart
<a></a><a></a><a></a><a></a></h2class="hidden"></h2class="hidden">0