403, 301, 302, 404 errors & possible google penalty
-
William Rock ran a Xenu site scan on
nlpca(dot)com
and mentioned the following:
...ran a test with Xenu site scan and it found a lot of broken links with 403, 301, 302, 404 Errors.
Other items found:
Broken page-local links (also named 'anchors', 'fragmentidentifiers'): http://www.nlpca.com/DCweb/Interesting_NLP_Sites.html#null anchor occurs multiple timeshttp://www.nlpca.com/DCweb/Interesting_NLP_Sites.html#US not found
Could somone give us an output of that list, and which ones of these errors do we need to clean up for SEO purposes?
Thank you.
-
Awesome, thanks Istvan and Andrea!
-
I agree - try Screaming Frog. I use it - and prefer it to Xenu - and it's free for up to 500 URLs or $99 for a license for unlimited. Even with a slower computer it can run in the background (leave it on overnight).
-
Hi Bob,
There is another tool which is developed for Mac too. It is called Screaming frog.
With that you can also check the website health just as with Xenu.
I hope it will be helpful.
Istvan
-
Hi Andrea,
I'm on a mac with a slow Windows Emulator. I was hoping someone could show me the URLs that need fixing seen by running the Xenu link test of
nlpca(dot)com
-
I can't quite follow - output from what?
Are you asking someone else to run a Xenu/Screaming frog of this URL (http://www.nlpca.com/DCweb/Interesting_NLP_Sites.html#null anchor occurs multiple timeshttp://www.nlpca.com/DCweb/Interesting_NLP_Sites.html#U) and give you feedback on what to fix (beyond the obvious that any 400 and 500 level errors should be tackled first)?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does non-critical AMP errors prevent you from being featured on Top Stories Carousel?
Consider site A which is a news publishing site that has valid AMP pages with non-critical AMP pages (as notified within Search Console). Also, Site A publishes news articles from site B (its partner site) and posts it on site A which have AMP pages too but most of them are not valid AMP pages with critical AMP errors. For brand terms like Economic Times, it does show a top stories carousel for all articles published by Economic Times, however it doesn't look the same for site A (inspite of it having valid AMP pages). Image link: http://tinypic.com/r/219bh9j/9 Now that there are valid AMP pages from site A and invalid AMP pages from site B on site A, there have been instances wherein a news article from site A features on the top stories carousel on Desktop for a certain query whereas it doesn't feature on the mobile SERPs in spite of the page being a valid AMP page. For example, as mentioned in the screenshot below: Business Today ranks on the Top Stories carousel for a term like “jio news” on Desktop, but on Mobile although the page is a valid AMP page, it doesn’t show as an AMP page within the Top Stories Carousel. Image Link: http://tinypic.com/r/11sc8j6/9 There have been some cases where although the page is featured on the top carousel on desktop, the same article doesn't show up on the mobile version for the same query on the Top Stories Carousel. What could be the reason behind this? Also, would it be necessary to solve both critical and non-critical errors on site A (including those published from site B on site A)?
Intermediate & Advanced SEO | | Starcom_Search1 -
I have 6 URL errors in GSC showing a 500 error code. How do I fix?
I am not sure how to fix some errors that are popping up in Google Search Console. The response codes showing are all: 500 error code I need some advice as to how to fix these. What are my options?
Intermediate & Advanced SEO | | pmull0 -
Can an incorrect 301 redirect or .htaccess code cause 500 errors?
Google Webmaster Tools is showing the following message: _Googlebot couldn't access the contents of this URL because the server had an internal error when trying to process the request. These errors tend to be with the server itself, not with the request. _ Before I contact the person who manages the server and hosting (essentially asking if the error is on his end) is there a chance I could have created an issue with an incorrect 301 redirect or other code added to .htaccess incorrectly? Here is the 301 redirect code I am using in .htaccess: RewriteEngine On RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/.]+/)*(index.html|default.asp)\ HTTP/ RewriteRule ^(([^/.]+/)*)(index|default) http://www.example.com/$1 [R=301,L] RewriteCond %{HTTP_HOST} !^(www.example.com)?$ [NC] RewriteRule (.*) http://www.example.com/$1 [R=301,L] Could adding the following code after that in the .htaccess potentially cause any issues? BEGIN EXPIRES <ifmodule mod_expires.c="">ExpiresActive On
Intermediate & Advanced SEO | | kimmiedawn
ExpiresDefault "access plus 10 days"
ExpiresByType text/css "access plus 1 week"
ExpiresByType text/plain "access plus 1 month"
ExpiresByType image/gif "access plus 1 month"
ExpiresByType image/png "access plus 1 month"
ExpiresByType image/jpeg "access plus 1 month"
ExpiresByType application/x-javascript "access plus 1 month"
ExpiresByType application/javascript "access plus 1 week"
ExpiresByType application/x-icon "access plus 1 year"</ifmodule> END EXPIRES (Edit) I'd like to add that there is a Wordpress blog on the site too at www.example.com/blog with the following code in it's .htaccess: BEGIN WordPress <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /blog/
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /blog/index.php [L]</ifmodule> END WordPress Thanks0 -
Webmaster tools 404
Hey, I'm getting a soft 404 error on a webpage that has content and is deferentially not a 404. We've redirect a load of urls to the web page. The url has parameters which was used before the redirect but are no longer used on by the new url, these parameters have been carried over in the redirect. Is this whats causing the soft 404 error or is there another problem that may need addressing? Also a canonical has been set on the webpage. Thanks, Luke.
Intermediate & Advanced SEO | | NoisyLittleMonkey1 -
Geoip redirection, 301 or 302?
Hello all Let me first try to explain what our company does and what it is trying to achieve. Our company has an online store, sells products for 3 different countries, and two languages for each country. Currently we have one site, which is open to all countries, what we are trying to achieve is make 3 different stores for these 3 different countries, so we can have a better control over the prices in each country. We are going to use Geoip to redirect the user to the local store in his country. The suggested new structure is to add sub-folders as following: www.example.com/ca-en
Intermediate & Advanced SEO | | ajarad
www.example.com/ca-fr
www.example.com/us-en
... If a visitor is located outside these 3 countries, then she'll be redirected to the root directory www.example.com/en We can't offer to expand our SEO team to optimize new pages for the local market, it's not the priority for now, the main objective now is to be able to control the prices for different market. so to eliminate the duplicate issue, we'll use canonical tags. Now knowing our objective from the new URL structure, I have two questions: 1- which redirect should we use? 301, 302?
If we choose 301, then which version of the site will get the link juice? (i.e, /ca-en or /us-en?)
if we choose 302, then will the link juice remain in the original links? is it healthy to use 302 for long term redirections? 2- Knowing that Google bots comes from US-IP, does that mean that the other versions of the site won't be crawled (i.e, www.example.com/ca-fr), this is especially important for us as we are using AdWords, and unindexed pages will effect our quality score badly. I'd like to know if you have other account structure in your mind that would be better than this proposed structure. Your help is highly highly appreciated.
Thanks in advance.0 -
Why will Google not remove a manual penalty against us?
Our site was placed under a manual penalty last year in June 2012 after penguin rolled out. We were advised by Google that we had unnatural links pointing to our site. We fought for months, running backlink checks and contacting webmasters where Google's WMT was showing the sites which had links. We have submitted numerous reconsideration requests with proof of our efforts in the form of huge well labeled spreadsheets, emails, and screen shots of online forms requesting link removal.When the disavow tool came out we thought it was a godsend and added all the sites who had either ignored us or refused to take down the links to the disavow.txt with the domain: tag. Then we submitted another reconsideration request, but to no avail.We have since had email correspondence with a member of the Google Quality Search Team who after reviewing the evidence of all our previous reconsideration requests and disavow.txt still advised us to make a genuine effort and listed sites which had inorganic links pointing to our site which were already included in the disavow.txt.Google has stated "In order for your site to have a successful reconsideration request, we will need to see a substantial, good-faith effort to remove the links, and this effort should result in a significant decrease in the number of bad links that we see."We have truly done everything we can and proven it too! Especially with all the sites in the disavow.txt there must be a decrease in links pointing to our site. What more can we do? Please help!
Intermediate & Advanced SEO | | Benbug0 -
Soft 404 problem
I have a soft 404 problem in webmaster tools for http://www.musicliveuk.com/about/feed and I'm not sure why. I read on here that if it is a main content page it should be fixed but I don't know how. I've tried to 301 redirect the page to http://www.musicliveuk.com/about/ but the redirect doesn't appear to be working? how do I fix this?
Intermediate & Advanced SEO | | SamCUK0
