How to find 20 hidden 404s
-
Hello,
We have like twenty 404s left to find. How do you find these when:
1. They don't show up in Google Webmaster Tools
2. They don't have any other internal or external pages linking to them.
3. They don't show up in site:domain.com (We have 9000 pages and only 600 show up - I fixed those out of the 600).
4. They are probably causing high bounce rates.
5. They're not in the sitemap
Thanks!
-
You should be able to crawl the entire site without increasing the RAM, once you buy the paid version.
-
Right in the paid version wont stop! You can see in their website that the limit of urls will be remove once you buy the license. http://www.screamingfrog.co.uk/seo-spider/licence/
It should catch everything, maybe you can contact their support just to make sure, they are very good on support over twitter.
-
Yes, but if it only check 397 in the free version is it going to stop there in the paid version as well. Just making sure.
Also, will it catch everything?
-
Yes I just run a craw for 500 000 urls
-
The free version checked 397 URLs. Will the purchased version check all 9000?
-
You can try crawl the site using screamingfrog, make sure you check the box “Check Links Outside Folder” in the spider menu.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Old URLs that have 301s to 404s not being de-indexed.
We have a scenario on a domain that recently moved to enforcing SSL. If a page is requested over non-ssl (http) requests, the server automatically redirects to the SSL (https) URL using a good old fashioned 301. This is great except for any page that no longer exists, in which case you get a 301 going to a 404. Here's what I mean. Case 1 - Good page: http://domain.com/goodpage -> 301 -> https://domain.com/goodpage -> 200 Case 2 - Bad page that no longer exists: http://domain.com/badpage -> 301 -> https://domain.com/badpage -> 404 Google is correctly re-indexing all the "good" pages and just displaying search results going directly to the https version. Google is stubbornly hanging on to all the "bad" pages and serving up the original URL (http://domain.com/badpage) unless we submit a removal request. But there are hundreds of these pages and this is starting to suck. Note: the load balancer does the SSL enforcement, not the CMS. So we can't detect a 404 and serve it up first. The CMS does the 404'ing. Any ideas on the best way to approach this problem? Or any idea why Google is holding on to all the old "bad" pages that no longer exist, given that we've clearly indicated with 301s that no one is home at the old address?
Intermediate & Advanced SEO | | boxclever0 -
How to find the redirects on website
I want to find the complete internal redirects on website. Just internally linked. How to find such?
Intermediate & Advanced SEO | | vtmoz0 -
Is there a good tool for finding co-occurance words/terms?
It would be great if there was a tool that helped find words/terms frequently used with other terms. For example, a search in this theoretical tool for "mobile app development" could bring up "iPhone app development" and "mobile app design" as examples of frequently co-occuring words. Any ideas or tools for this kinda thing?
Intermediate & Advanced SEO | | jim_shook1 -
How do you find all of your 301 redirects?
I would like a full list of 301 redirects that we have on our site. Is there a way to export it? I tried Bing Webmaster tools and the list was incomplete. We use IIS. Any ideas?
Intermediate & Advanced SEO | | EcommerceSite0 -
Where is the best place to find an SEO coach?
I'm looking for a coach who can help me get to the next level with SEO and help me determine what's junk and what's true advice. Are there any recommendations any of you have out there on where to find such a person?
Intermediate & Advanced SEO | | kadesmith0 -
Is there a way to find out how many 301 redirects a site gets?
If you do a search on "personal loans" on Google the first non-local/personal result is onemainfinanical.com. They have far fewer links showing in OSE and YSE than the other sites. I know onemainfinanical.com is a Citbank site so I'm trying to determine if they are ranking so high b/c they are getting 301 link juice from old Citibank.com authority pages. Is there anyway to check to see what sites are sending link juice through a 301 redirect instead of a direct link?
Intermediate & Advanced SEO | | fthead90 -
Best way to find all url parameters?
In reference to http://googlewebmastercentral.blogspot.com/2011/07/improved-handling-of-urls-with.html, what is the best way to find all of the parameters that need to be addressed? Thanks!
Intermediate & Advanced SEO | | nicole.healthline0 -
Hidden Text? Part of the game or a major faux pas?
Hey all... I've been doing some competitive analysis lately on different niches in different cities, and upon looking at source code have noticed MANY sites(typically 2-3 of the top listed sites in each niche) who have hidden text, blocked by a box/logo, colored the same as the background, or both. Is this a typical practice? My understanding is Google seriously frowns on this, so is it worth reporting competitors who are doing such things? I don't see any grey areas in this matter and think it's a serious violation, but here these sites are at the top.
Intermediate & Advanced SEO | | dignan990