Increased 404 and Blocked URL Notifications in Webmaster Tools
-
In the last 45 days, I am receiving an increasing number of 404 alerts in Google Webmaster Tools.
When I audit the notifications, they are not "new" broken links, these are all links that have been pointing to non-existent pages for years that for some reason Google is just notifying me about them. This has also coincided with about a 30% drop in organic traffic from late April to early May.
The site is www.petersons.com and its been around for a while and the site attracts a fair amount of natural links so in the 2 years I've managed the campaign I've done very little link-building.
I'm in the process of setting up redirects for these urls but why is Google now notifying me of years old broken links and could that be one of the reasons for my drop in traffic.
My second issue is my I am being notified that I am blocking over 8,000 urls in my Robots file when I am not. I attached a screenshot.
Here is a link to a screenshot. http://i.imgur.com/ncoERgV.jpg
-
I doubt very much that an increase in old 404s resulted in a 30% organic traffic drop. I'd look closely at your backlink profile, competition and page quality to try and diagnose why you saw that drop in traffic.
As for the 404s I'd fix those that are fixable and 301 redirect the rest to relevant pages (or the home page). If the number is extremely large then you should put a high priority on fixing this. Otherwise I haven't met a site that Google couldn't find a 404 error on. And yeah, they keep telling you about the same ones!
Hope that helps!
Jacob
-
Hi!
As Lynn points out, there could be some issues in regards to your perceived uptime. Do you see a lot of 404 errors reported in Analytics as well? If this is the case, perhaps your hosting provider (or IT department) should have a look at this?
Also, adding the redirects seems like a good idea, as Google couold be reindexing some sites/pages linking to the old, deleted URL's.
Do you have a custom crawl frequency set up in Google Webmaster Tools? It's worth looking into if Googlebot is slowing down your site.
Good luck.Anders
-
The Moz scan is not showing the same errors. And we haven't made any technological changes. These are incoming links pointing to pages that don't exist anymore. It looks like its been that way for years, I just started getting notified of these and I'm wondering if somehow it is hurting the site.
About the robots file, I just don't know. I've decided to make it blank and re-assess in a few days.
-
Hi,
A bit difficult to say without some more details. Some of it might be outdated information. See: http://moz.com/blog/how-to-fix-crawl-errors-in-google-webmaster-tools for a rundown on how to check it if you haven't already. What urls is it flagging from the robots.txt? Are they still valid urls? In regards the 404s, 28,000 is quite a lot. Has your system changed or been updated recently? Maybe there is a systemic fault going on that is creating these errors? Is the moz scan flagging the same errors?
It is tough to say if the errors have any connection to the drop in visits, but it is certainly something you want to get to the bottom of. I threw your site into xenu (http://home.snafu.de/tilman/xenulink.html) and it was timing out on quite a few of the pages. Is it possible the site is timing out on heavy loads? That might account for the drop in organic visits also...
Lots of questions, not many answers!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What kind of impact does a 404 have in a sitemap regarding ranking?
We recently had a site update where our robots file disallowed our sitemap for about two weeks. When we found the problem and resubmitted the sitemap to Google Search Console, it found a 404 error. Does this have any impact on ranking or visibility if we are still recovering from the disallow?
Algorithm Updates | | GaryBlanchard0 -
Meta robots at every page rather than using robots.txt for blocking crawlers? How they'll get indexed if we block crawlers?
Hi all, The suggestion to use meta robots tag rather than robots.txt file is to make sure the pages do not get indexed if their hyperlinks are available anywhere on the internet. I don't understand how the pages will be indexed if the entire site is blocked? Even though there are page links are available, will Google really index those pages? One of our site got blocked from robots file but internal links are available on internet for years which are not been indexed. So technically robots.txt file is quite enough right? Please clarify and guide me if I'm wrong. Thanks
Algorithm Updates | | vtmoz0 -
How do you get a url to show as a tagline in google mobile search?
When searching in google via mobile, I am seeing urls changed to taglines. I have attached pictures that show the url in a web search, but a tag line from the mobile search. Does anyone know how to get a tagline to show in place of a url in a mobile search? Any advice would be appreciated! uLkYWRx.png wljXRI3.png
Algorithm Updates | | David-Kley0 -
Key Word in URL - To Include or Exclude?
Hi MoZ Community, Key word inclusion in URL has been discussed a fair bit on here and curious for some feedback on two options on URL structure. Ran’s #3 tip from his recent ‘15 SEO Best Practices for Structuring URLs’ states that key word inclusion still has some value but I’m not too sure if we’re going too far with the below examples. We sell footwear and only footwear for Women, Men & Kids and use those words as our key menu headings at the top. Under each of the main headings within a mega menu the users then has the choice to ‘shop by style’, ‘shop by brand’ etc… The key question or feedback is about including the word ‘shoes’ in my URLs as many of the top ranking competitors do it. e.g. /women-shoes-heels, womens-shoes-sandals or womens-shoes/heels, womens-shoes/sandals I think Google is smart enough to determine we have a shoe store and not sure of the value from a SEO or user experience perspective of adding the additional word. Thoughts on going with option A or B would be valued.... Option A - http://shopname.com/womens/sandals, http://shopname.com/womens/heels OR Option B - http://shopname.com/womens-shoes/sandals, http://shopname.com/womens-shoes/heels Thanks, | | |
Algorithm Updates | | chewythedog
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |0 -
Flat Structure URL vs Structured Sub-directory URL
We are finally taking our classifieds site forward and moving into a much improved URL structure, however, there is some disagreement over whether to go with a Flat URL structure or a structured sub-directory. I've browsed all of the posts and Q&A's for this going back to 2011, and still don't feel like I have a real answer. Has anyone tested this yet, or is there any consensus over ranking? I am in a disagreement with another SEO manager about this for our proposed URL structure redesign who is for it because it is what our competitors are doing. Our classifieds are geographically based, and we group by state, county, and city. Most of our traffic comes from state and county based searches. We also would like to integrate categories into the URL for some of the major search terms we see. The disagreement arises around how to structure the site. I prefer the logical sub-directory style: [sitename]/[category]/[state]/[county]/
Algorithm Updates | | newspore
mysite.com/for-sale/california/kern-county/
or
[sitename]/[category]/[county]-county-[stateabb]/
mysite.com/for-sale/kern-county-ca/ I don't mind the second, except for when you look at it in the context of the whole site: Geo Landing Pages:
mysite.com/california/
mysite.com/los-angeles-ca-90210/ Actual Search Pages:
mysite.com/for-sale/orange-ca/[filters] Detail Pages:
mysite.com/widget-type/cool-product-name/productid I want to make sure this flat structure performs better before sacrificing my analytics sanity (and ordered logic). Any case studies, tests or real data around this would be most helpful, someone at Moz must've tackled this by now!0 -
How Additional Characters and Numbers in URL affect SEO
Hi fellow SEOmozers, I noticed that a lot of websites have additional characters and words at the end of the URL in addition keyword optimized URL. Mostly for E-Commerce sites For example: www.yoursite.com/category/keyword?id=12345&Keyword--Category--cm_jdkfls_dklj or wwww.yoursite.com/category/keyword#83939=-37292 My question is how does the additional characters or parameters(not necessarily tracking parameters) affect SEO? Does it matter if i have additional keywords in the additional stuff in the URL (1st url example)? If you can provide more information, that would be helpful. Thank you!
Algorithm Updates | | TommyTan0 -
How to keep damage low on Google after the change of URL's
Hi Peeps, Hope someone can shed a light on this and show a guidance if possible. We are going to move our sites to shopify and shopify's URL's cannot be customized to match exactly like our current URLs. What steps do I need to take so google knows the URL's are changed. Domain will be the same. Thank you in advanced.
Algorithm Updates | | cemalcebi0 -
The related: query for one of my urls makes no sense
I'm trying to compete regarding keyword X. Currently, I'm on first page, 7-8th position. If, for each one of the urls listed in first page for such keyword, I search for related:[url], I get similar results for all of them, but mine. Mine shows inconsistent results, none of which related to the same topic as the other 9 in the top 10. Looking at them, the only hypothesis I am able to formulate is that, somehow, google is linking the url to its paid banners in big media. However, such banners go through an adserver and/or are declared as nofollow. Is there any obvious reason that could be causing this? I wonder if we are on page 1 even though we're considered pretty-much 'off-topic' regarding the keyword.
Algorithm Updates | | jleanv240