How to identify 404 that get links from external sites (but not search engines)?
-
one of our site had a poor site architecture causing now about 10.000s of 404 being currently reported in google webmaster tools.
-
Any idea about easily detecting among these thousands of 404, which ones are coming from links from external websites (so filtering out 404 caused by links from our own domain and 404 from search engines)?
-
crawl bandwidth seems to be an issue on this domain. Anything that can be done to accelerate google removing these 404 pages from their index? Due to number of 404 manual submission in google wbt one by one is not an option.
Or do you believe that google automatically will stop crawling these 404 pages within a month or so and no action needs to be taken?
thanks
-
-
Hi Robert,
thanks a lot. So I will not take action to get 404s out of google index.
Regarding your first point, I am not sure I understand how screaming frog would help. I did not use screaming frog yet but link sleuth for status code checks. The status check of 404 in google webmaster tools will probably generally also give 404 status in screaming frog. My objective is to identify among these thousands of 404, the few which are caused by inaccurate or outdated links on external websites so that I can create a 301 for these.
Best,
Daniel
-
icourse
I would suggest downloading the free version of screaming frog for an easy way to get status codes on any or all links.
As to fixing and "crawl bandwidth" being a problem, I disagree. If you are not being crawled it is because of all the 404's. I do not know the timeline for inaction on this, but I do believe "manual submission is not an option" is a recipe for disaster. Because fully analyzing your issues is outside the scope of Q&A, I would suggest you start manually fixing the issues and if on a CMS, start looking at plugins, etc. as a root cause.
Hope that helps
Robert
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Importance of external links in 2018
How important are external links in 2018. How much of a percentage do they represent when deciding to rank a page. I imagine it depends on the query but I was wondering it if 10 % of of 60 % ? My feeling is that with good content you can get on almost any query on the 1 st page without links because that would be too penalising to small business if they had no possibility to rank with just content. Looking forward to getting some feedback.
Intermediate & Advanced SEO | | seoanalytics2 -
Getting Rid Of Spammy 301 Links From An Old Site
A relatively new site I'm working on has been hit really hard by Panda, due to over optimization of 301 external links which include exact keyword phrases, from an old site. Prior to the Panda update, all of these 301 redirects worked like a charm, but now all of these 301's from the old url are killing the new site, because all the hyper-text links include exact keyword matches. A couple weeks ago, I took the old site completely down, and removed the htaccess file, removing the 301's and in effect breaking all of these bad links. Consequently, if one were to type this old url, you'd be directed to the domain registrar, and not redirected to the new site. My hope is to eliminate most of the bad links, that are mostly on spammy sites, that aren't worth linking to. My thought is these links would eventually disappear from G. My concern is that this might not work, because G won't re-index these links, because once they're indexed by G, they'll be there forever. My fear is causing me to conclude I should hedge my bets, and just disavow these sites using the disavow tool in WMT. IMO, the disavow tool is an action of last resort, because I don't want to call attention to myself, since this site doesn't have a manual penalty inflected on it. Any opinions or advise would be greatly appreciated.
Intermediate & Advanced SEO | | alrockn0 -
Can you recover from "Unnatural links to your site—impacts links" if you remove them or have they already been discounted?
If Google has already discounted the value of the links and my rankings dropped because in the past these links passed value and now they don't. Is there any reason to remove them? If I do remove them, is there a chance of "recovery" or should I just move forward with my 8 month old blogging/content marketing campaign.
Intermediate & Advanced SEO | | Beastrip0 -
On site links triggering anchor text algorithmic penatly?
I'm trying to figure out why a drop in ranking occurred and think it may be related to an increase in on site links. I've attached images of the SEO moz report showing a jump in links from a few hundred to around 15,000 within the space of a week. I think this may be due to some on site work I did when I created categories (I use wordpress) for a large number of cities and towns in the UK. I soon realised I'd run into duplicate content issues and removed all these categories within a few days. As I added categories I also ran into 'too many on page links' warnings as each category I added created a new link and I ended up with hundreds on each page. If you look at the analytics reports I suffered a huge drop in rankings on the 10th March and think this could be due to an on site anchor text problem that was caused by adding the categories and in turn creating many on site links. SEO moz found these links on the 11th and 25th Feb but my guess is that Google found them around at the same time but if these links are the problem then why didn't my rankings drop until the 10th March? Surely they would have dropped sooner? Would this cause a drop in rankings? I've recieved an email from google saying that no manual penalty was applied to the site after I submitted a reconsideration request. Therefore it must be some kind of algorithmic penalty. Could this be the problem and if not what else should I look at. My baclink profile appears to be okay and I've been careful to vary my anchor text with inbound link building. I'm at a loss as to what to do next. Any help will be much appreciated! UXsMLYS.png Ov9AOs8.png
Intermediate & Advanced SEO | | SamCUK0 -
How to cross-link a network of dozen hotel booking sites
Hi, I have about a dozen hotel booking sites in different cities. All content is unique on each site and specific to its city. Is it a good idea to cross-link them all? If so, is there any strategy to follow? Some of them are older and well established sites wheres others were created recently. The goal is to improve rankings of the newer sites. I appreciate any advice. Thanks!
Intermediate & Advanced SEO | | vmotuz0 -
Purchased new site with good SERP ranks, do I operate and build links or redirect the TLD?
I recently purchased a blog within my product category - it has many first page rankings for difficult keywords within my niche. I am wondering if it makes more sense for for me to continue to operate this blog and build links to my site and blog (blog is in wordpress) or to export the XML feed and upload the content to my blog (new site also in wordpress), at which point I would do a 301 at the Top-Level domain. Any thoughts, ideas, or personal experiences would be greatly appreciated. Thanks!
Intermediate & Advanced SEO | | NickEubanks0 -
Migrating multiple sites and trying to save link juice
I have an interesting problem SEOmozers and wanted to see if I could get some good ideas as to what I should to for the greatest benefit. I have an ecommerce website that sells tire sensors. We just converted the old site to a new platform and payment processor, so the site has changed completely from the original, just offering virtually the same products as before. You can find it at www.tire-sensors.com We're ranked #1 for the keyword "tire sensors" in Google. We sell sensors for ford, honda, toyota, etc -- and tire-sensors.com has all of those listed. Before I came along, the company I'm working for also had individual "mini ecommerce" sites created with only 1 brand of sensors and the URL to match that maker. Example : www.fordtiresensors.com is our site, only sells the Ford parts from our main site, and ranks #1 in Google for "ford tire sensors" I don't have analytics on these old sites but Google Keyword Tool is saying "ford tire sensors" gets 880 local searches a month, and other brand-specific tire sensors are receiving traffic as well. We have many other sites that are doing the same thing. www.suzukitiresensors.com (ranked #2 for "suzuki tire sensors") Only sells our Suzuki collection from the main site's inventory etc We need to get rid of the old sites because we want to shut down the payment gateway and various other things those sites are using, and move to one consolidated system (aka www.tire-sensors.com) Would simply making each maker-specific URL (ie. fordtiresensors.com) 301 redirect to our main site (www.tire-sensors.com) give us to most benefit, rankings, traffic etc? Or would that be detrimental to what we're trying to do -- capturing the tire sensors market for all car manufacturers? Suggestions? Thanks a lot in advance! Jordan
Intermediate & Advanced SEO | | JordanGodbey0 -
Do search engines understand special/foreign characters?
We carry a few brands that have special foreign characters, e.g., Kühl, Lolë, but do search engines recognize special unicode characters? Obviously we would want to spend more energy optimizing keywords that potential customers can type with a keyboard, but is it worthwhile to throw in some encoded keywords and anchor text for people that copy-paste these words into a search? Do search engines typically equate special characters to their closest English equivalent, or are "Kuhl", "Kühl" and "Kühl" three entirely different terms?
Intermediate & Advanced SEO | | TahoeMountain400