Google Webmaster Tools reports 404s for plain text. Should I create those URLs and 301 them to actual pages?
-
IA few years back I noticed that Google Webmaster Tools returns 404s from regular text containing a URL, but no anchor tag. I came accross this again today. Is it worthwhile to create those URLs and 301 redirect them to proper pages.
-
Hi Svetoslav,
Great question because I saw exactly something like this a couple of weeks ago in our own Google Webmaster Tools. Google tried to crawl "urls" on our side which were used to generated pageviews for Google Analytics by using the _trackPageview. So we had a lot of urls like "/print" within our Not Found crawl errors.
I won't recommend creating new URLs and using a 301 redirect to "cover" them, for a couple of reasons:
- From my opinion the Not Found errors do not harm you're rankings within the SERPs, to give you an idea or maybe you experienced it yourself. We have ten thousands of Not Found errors because our content changes on a daily basis and we saw it makes no sense.
- Google is trying to crawl urls which don't exist because they are text, so it looks like to me they are doing something wrong.
- And I also would like to share this thread from WebmasterWorld ( ), it includes an interesting angle in my opinion: "S_ave the 301 (and its machine process) for important cases"._
What do you think?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to 301 trailing URLs to new domain home page - wildcard?
How would I add a redirect rule so all old domain URLs redirect to a new domain? All the old pages no longer exist on a new website. The domains have been through several CMS platforms, so it would be unnecessary to recreate them. Problem is, they're indexed in search engines from the past 10 years, so it's causing a lot of 404s. Example: search "NARI Tampa Bay" and you'll find 2 old domains: nari-tampabay.com & nari-tampabay.org. The new domain is naritb.org Those 2 old domains are now pointed to the same nameservers as the new and listed as parked domains. Here's the current rules in htaccess: <code>RewriteEngine On RewriteCond %{HTTP_HOST} ^nari-tampabay.org [NC,OR] RewriteCond %{HTTP_HOST} ^www.nari-tampabay.org [NC] RewriteRule ^(.*)$ https://www.naritb.org/$1 [L,R=301] RewriteEngine On RewriteCond %{HTTP_HOST} ^nari-tampabay.com [NC,OR] RewriteCond %{HTTP_HOST} ^www.nari-tampabay.com [NC] RewriteRule ^(.*)$ https://www.naritb.org/$1 [L,R=301]</code>
Technical SEO | | CartoMark0 -
Using the Google Remove URL Tool to remove https pages
I have found a way to get a list of 'some' of my 180,000+ garbage URLs now, and I'm going through the tedious task of using the URL removal tool to put them in one at a time. Between that and my robots.txt file and the URL Parameters, I'm hoping to see some change each week. I have noticed when I put URL's starting with https:// in to the removal tool, it adds the http:// main URL at the front. For example, I add to the removal tool:- https://www.mydomain.com/blah.html?search_garbage_url_addition On the confirmation page, the URL actually shows as:- http://www.mydomain.com/https://www.mydomain.com/blah.html?search_garbage_url_addition I don't want to accidentally remove my main URL or cause problems. Is this the right way this should look? AND PART 2 OF MY QUESTION If you see the search description in Google for a page you want removed that says the following in the SERP results, should I still go to the trouble of putting in the removal request? www.domain.com/url.html?xsearch_... A description for this result is not available because of this site's robots.txt – learn more.
Technical SEO | | sparrowdog1 -
Help creating a 301 redirect in my htaccess file
Hi Guys, I'm trying to build a 301 file with the file requirements: It should be visible only for Google and other Search Engine Agents. It will have a few direct redirects. A few URL must be dynamic redirect. For example each page the starts with olddomain.com/category and is not in the list of of direct redirects should be redirect for newdomain.com/category Here is my start point: #301 Starts here Set the agents RewriteEngine On
Technical SEO | | Felip3
RewriteCond %{HTTP_USER_AGENT} Googlebot [OR]
RewriteCond %{HTTP_USER_AGENT} msnbot [OR]
RewriteCond %{HTTP_USER_AGENT} Slurp Make the direct redirect. Redirect 301 /category/sub1 http://www.newdomain.com/category/sub1
Redirect 301 /category/sub2 http://www.newdomain.com/category/sub2 Redirect everything else Redirect 301 /category/* http://www.newdomain.com/category #End of my 301 Will that work how I want? is there anything wrong?0 -
Strange Webmaster Tools Crawl Report
Up until recently I had robots.txt blocking the indexing of my pdf files which are all manuals for products we sell. I changed this last week to allow indexing of those files and now my webmaster tools crawl report is listing all my pdfs as not founds. What is really strange is that Webmaster Tools is listing an incorrect link structure: "domain.com/file.pdf" instead of "domain.com/manuals/file.pdf" Why is google indexing these particular pages incorrectly? My robots.txt has nothing else in it besides a disallow for an entirely different folder on my server and my htaccess is not redirecting anything in regards to my manuals folder either. Even in the case of outside links present in the crawl report supposedly linking to this 404 file when I visit these 3rd party pages they have the correct link structure. Hope someone can help because right now my not founds are up in the 500s and that can't be good 🙂 Thanks is advance!
Technical SEO | | Virage0 -
Why is google not deindexing pages with the meta noindex tag?
On our website www.keystonepetplace.com we added the meta noindex tag to category pages that were created by the sorting function. Google no longer seems to be adding more of these pages to the index, but the pages that were already added are still in the index when I check via site:keystonepetplace.com Here is an example page: http://www.keystonepetplace.com/dog/dog-food?limit=50 How long should it take for these pages to disappear from the index?
Technical SEO | | JGar-2203710 -
Google places page where is my additional information
Hi When creating Google places you can add additional information but where does this information go? its not showing up on the page when you place page? Whats the best practice when creating pages in relation to optimising them ? thanks
Technical SEO | | Bristolweb0 -
Should we block URL param in Webmaster tools after URL migration?
Hi, We have just released a new version of our website that now has a human readable nice URL's. Our old ugly URL's are still accessible and cannot be blocked/redirected. These old URL's use a URL param that has an xpath like expression language to define the location in our catalog. We have about 2 million pages indexed with this old URL param in it while we have approximately 70k nice URL's after the migration. This high number of old URL's is due to facetting that was done using this URL param. I wonder if we should now completely block this URL param from Google Webmaster tools so that these ugly URL's will be removed from the Google index. Or will this harm our position in Google? Thanks, Chris
Technical SEO | | eCommerceSEO0 -
301 an old URL with a ? in the URL?
I am redoing a site and the URL's are changing structure. The client's site was in magento and in the store they would get two URLs, for example: /store/categoryname/productname and /store/categoryname/productname?SID=dslkajsfdoiu947598whouieht983hg98 Do I have to 301 redirect both of these URL's to their new counterpart? Both go to the same content but magento seemed to add these SIDs into the navigation and Google has both versions in the index.
Technical SEO | | DanDeceuster0