404s in Google Search Console and javascript
-
The end of April, we made the switch from http to https and I was prepared for a surge in crawl errors while Google sorted out our site. However, I wasn't prepared for the surge in impossibly incorrect URLs and partial URLs that I've seen since then.
I have learned that as Googlebot grows up, he'she's now attempting to read more javascript and will occasionally try to parse out and "read" a URL in a string of javascript code where no URL is actually present. So, I've "marked as fixed" hundreds of bits like
/TRo39,
category/cig
etc., etc....But they are also returning hundreds of otherwise correct URLs with a .html extension when our CMS system generates URLs with a .uts extension like this:
https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.html
when it should be:
https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.utsWorst of all, when I look at them in GSC and check the "linked from" tab it shows they are linked from themselves, so I can't backtrack and find a common source of the error.
Is anyone else experiencing this? Got any suggestions on how to stop it from happening in the future? Last month it was 50 URLs, this month 150, so I can't keep creating redirects and hoping it goes away.
Thanks for any and all suggestions!
Liz Micik -
Hi Liz,
What I would do as well is go with a solution around your robots.txt to make sure the crawlers will respect it and don't go on a hunch trying to find new URLs that are embedded somewhere else. Usually it's something you shouldn't worry about too much, it's just the crawler doing a good job trying to find more content/URLs on your site.
Martijn.
-
Google Search Console 404s can be very irritating.
What does your robots.txt file currently say?
You can remove all instances of .html in your htaccess file to help solve for this issue. More on that here: https://alexcican.com/post/how-to-remove-php-html-htm-extensions-with-htaccess/
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google panda, penguin or Patience needed?
Dear friends, On 3rd of May, i suffered a Manual google unnatural outbound link penalty. I recovered from the said penalty on 27th of June. However, i have noticed that my traffic has been dropping since 23rd April. I am confused where to target my work. Should i work on thin content and is it an algorithmic Panda problem (but my keywords are still ranking good) or is it a Penguin problem (I had 6 domains with payday loans backlinks and i have dosavowed 32 backlinks recently) What should be my plan of action here and what would you recommend? An image is attached herewith for your reference, PHd8BzX.png
Algorithm Updates | | marketing911 -
Indexing of Search Pages
I have a question on indexing search pages of an ecommerce or any website. I read Google doesn't recommend this and sites shouldn't allow indexing of their search pages. I recently attended an SEO event (BrightonSEO) and one of the talks was on search pages and how big players like eBay, Amazon do index their search pages. In fact, it is a core part of the pages that are indexed. eBay has to do it, as their product pages are on a time frame and Amazon only allows certain category search pages to be indexed. Reviewing my competitors, they are indexing search pages and this is why they have thousands and millions of web pages indexed. What are your thoughts? I thought search pages were too dynamic (URL strings) and they wouldn't have a unique page title, meta description or rich content to act as a well optimised page. Am I missing a trick here? Cyto
Algorithm Updates | | Bio-RadAbs0 -
New visual search results - what is this and how do we optimize for it?
Hi all, This morning as I am doing some keyword research for a new client, I typed in the phrase and Google returned both the listings as well as a vertical photo bar. I've never seen this before. Is this new? Is it common and I've just missed it? I presume this means we need to really have our photo alt tags 'ducks in a row' but I'm also wondering if this points to an increased importance on visual content? Image attached. Thanks, YINd14d
Algorithm Updates | | EricOliver0 -
How come google image search doesn't link to the right page?
For one site I work with the images link to the home page of the site rather than the page the image lives on. I think this is hurting my bounce rate quite a bit. Thoughts?
Algorithm Updates | | NetvantageMarketing0 -
How to search for popular press releases
I would like to research popular press releases in my industry. Ones that got picked up by many popular outlets, got a lot of coverage etc. Besides mindlessly searching the web for press releases, is there a better way? Almost looking for a service that ranks press releases in terms of effectiveness.
Algorithm Updates | | StreetwiseReports0 -
How to Recover Ranking from Latest Google Panda and EMD Update?
We have 7 websites with the exact match domains. Our Website has been affected by Google recently updated (Panda and EMD Update). we don't want to do any changes in our existing domains. What should we do? So, What is the exact solution for that. Help Us out! Website Names: http://www.hamptoninndenverairport.com http://www.wingatehotelcolumbia.com http://www.fairfieldinnhotelaurora.com http://www.hamptoninnhotelcrestwood.com http://www.laquintahoteldavenport.com http://www.redroofinnhotelcedarrapids.com http://www.hamptoninnhotelcarolstream.com Thanks in advance.
Algorithm Updates | | CommercePundit1 -
Does Google do domain level topic modeling? If so, are off-site factors such as search traffic volume taken into account?
80% of my site's organic traffic is coming through a resource that is only somewhat related. Does Google think the main topic of my site is terms this resource targets thus bumping the terms I care about to a sub-topic level of sorts? If this is the case, would putting the resource information into a sub-domain help to solve the problem?
Algorithm Updates | | tatermarketing0 -
What determines rankings in a site: search?
When I perform a "site:" search on my domains (without specifying a keyword) the top ranked results seem to be a mixture of sensible top-level index pages plus some very random articles. Is there any significance to what Google ranks highly in a site: search? There is some really unrepresentative content returned on page 1, including articles that get virtually no traffic. Is this seriously what Google considers our best or most typical content?
Algorithm Updates | | Dennis-529610
