Huge number of crawl anomalies and 404s - non- existent urls
-
Hi there,
Our site was redesigned at the end of January 2020. Since the new site was launched we have seen a big drop in impressions (50-60%) and also a big drop in total and organic traffic (again 50-60%) when compared to the old site.
I know in the current climate some businesses will see a drop in traffic, however we are a tech business and some of our core search terms have increased in search volume as a result of remote-working.
According to search console there are 82k urls excluded from coverage - the majority of these are classed as 'crawl anomaly' and there are 250+ 404's - almost all of the urls are non-existent, they have our root domain with a string of random characters on the end. Here are a couple of examples:
root.domain.com/96jumblestorebb42a1c2320800306682
root.domain.com/01sportsplazac9a3c52miz-63jth601
root.domain.com/39autoparts-agency26be7ff420582220
root.domain.com/05open-kitchenaf69a7a29510363
Is this a cause for concern? I'm thinking that all of these random fake urls could be preventing genuine pages from being indexed / or they could be having an impact on our search visibility. Can somebody advise please?
Thanks!
-
Unlikely, as long as they're returning 404 errors you should be OK. Maybe update your disavow file and you should be good to go!
-
Thanks for your reply.
I’m new to the business and I’ve found that that the old website had a spam attack, all of these fake urls are from the old pages (as they have 301s).
There are 82,000 crawl anomalies from these fake/spam URLs and around 200 404s. None of the fake /spam urls have been indexed. Could this be having a negative effect of search visibility/DA or rankings?
Thanks!
-
It's tough to say without seeing the site. Overall it's unlikely if you don't use that string anywhere. We usually see it more for broken relative URLs. Maybe a third party site is using that string.
-
Thanks for your reply, would broken urls from the internal linking structure explain the random characters? e.g. root.domain.com/96jumblestorebb42a1c2320800306682
We've never had any page content/urls relating to 'jumblestore'.
Thanks!
-
From what I can tell, this probably isn't the reasons for the drops. I'd go back and ensure that any URLs that changed are 301 redirecting to the correct destination URL. I'd also ensure that no pages that were associated with high volume keywords no longer exist.
For your issue, Google is likely finding some broken URLs, possibly from your internal linking structure. Perform a crawl of the site and see if you can find "Inlinks" to those broken pages. If so, you can work with dev to eliminate the issue.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Which URL is better?
Hi everyone, Could you please help me with picking out the right URL for my company's website? We are MoonCreate and we make beautiful clothes. Unfortunately, the domain mooncreate.com is not available and I have to choose between mooncreatebrand.com or mooncreatewear.com Which one is better, in your opinion? Look forward to receive your suggestions! Thank you! 🙂
Technical SEO | | kirupa0 -
Woocommerce URL Structure Issue
Hi everyone ! To put you in context, I am doing an audit on an E-Commerce site selling auto parts with WooCommerce. I have some concerns regarding the url structure and here's why: Product category page url: /auto/drivetrain/cv-axle-shaft-assembly/
Technical SEO | | alexrbrg
Product page url included in the product category page: /product/acura-integra-cv-axle-shaft-90-01-honda-civic/ The way I see my situation is that the product page is considered by Google as an intern link and not as a page included in in the subfolder of the category page. 1. Am I right?
2. If yes, is there a solution to fix the issue with woocommerce to improve the category page ranking ? Thanks y'all !0 -
Crawl depth and www
I've run a crawl on a popular amphibian based tool, just wanted to confirm... should http://www.homepage be at crawl depth 0 or 1? The audit shows http://homepage at level 0 and http://www.homepage at level 1 through a redirect. Thanks
Technical SEO | | Focus-Online-Management0 -
URL Question: Is there any value for ecomm sites in having a reverse "breadcrumb" in the URL?
Wondering if there is any value for e-comm sites to feature a reverse breadcrumb like structure in the URL? For example: Example: https://www.grainger.com/category/anchor-bolts/anchors/fasteners/ecatalog/N-8j5?ssf=3&ssf=3 where we have a reverse categorization happening? with /level2-sub-cat/level1-sub-cat/category in the reverse order as to the actual location on the site. Category: Fasteners
Technical SEO | | ROI_DNA
Sub-Cat (level 1): Anchors
Sub-Cat (level 2): Anchor Bolts0 -
Formatting dynamic urls?
We have a long-time previously well-established website that was hit by panda. On one section of the site, we have dynamic urls that include %20 in them (e.g. North%20America). It's recently come to our attention that google has both a version of the url with a plus sign (+) and the version with the %20 (space) (e.g. North+America). Upon researching this, it seems that a hyphen (-) is preferable to either of the above. We obviously need to remove the %20's from the urls as they can cause issues. So, should we stick with the + sign since it's already indexed and ranking or do a 301 rewrite and change them all to hyphens instead of the plus sign? This is the one section of the site that has maintained rankings through the panda debacle, so we need to take that into consideration as we don’t want to lose the rankings that we have. Along the same lines, we have two other sections of the site that provide search results as well, though these are all formatted to use a plus sign. Is it advisable to do a 301 rewrite to change the plus signs to hyphens on these as well or just leave them alone? This particular section has lost rankings over the last year with panda updates.
Technical SEO | | Odjobob0 -
New URL or Folder Off Existing Site
I am working on a project that is promoting dining in a particular region of the southwest for a destination marketing company. The parent Web site is an authority in the region and ranks well for almost all terms related to the leisure experience in the region. A completely separate Web site was built to promote this culinary program as it involves a committee of different stakeholders, but it’s solely focused on the region. My question is this. The site is on a different CMS, etc., but the overall experience on the site is similar to the parent DMO site in terms of creative. The client has a brand new domain that they purchased for this initiative, but we are also considering mapping the parent site URL to the new culinary site. Parent: www.regionalsite.com New Themed Site: www.regionalsite.com/theme/ Or www.themeurl.com My fear is that if I take the approach of the new URL that it will take forever for the site to build any link clout at all, as the client doesn’t really get the fact that working a link strategy is so critical. However, I know that having links from the regional site over to the theme URL will have an impact. Also, if I do take the approach of mapping the URL to a new folder off of the parent domain, do I risk that 2<sup>nd</sup> tier links on the micro-site will have a challenge indexing as they will essentially be on tier 3? Any advice would be appreciated.
Technical SEO | | VERBInteractive0 -
Should I use www. or not in my main URL?
I have backlinks coming into my homepage, which has both a www. URL and one that's merely http://mysite.com. Which is the preferred URL for best optimization for search engines and how do I find this out?
Technical SEO | | NetPicks0 -
New URL structure caused a HUGE drop?
I have started working with a client who did an upgrade on their e-commerce sive in May of last year. It totally changed the URL structure and they didn't redirect old URLs or do any of the things they should have. Not unexpectedly they they went from about 300 visitors a day to 0 for then rose up to maybe 50 and have remained there ever since. There were some major onsite issues including about 15000 internal links that 302 back to the site. In any case I have fixed most of the onsite problems and worked on a little better categorization + content optimization, etc. We have only been working on this for about 30 days and organic traffic is up and they are ranking for much better keywords, but I expected a little quicker rise. Here is a screenshot out of GA of their descent. Its pretty rapid. I dont think it makes sense to redirect their old URLs at this point since most of them have been deindexed for 10+ months. Anyone have any suggestions on how to get back to their previous level. The domain actually has decent authority and link profile, etc. Is this just going to be a slow climb back? Any thoughts? Fxz9Y.png
Technical SEO | | BlinkWeb0