[Very Urgent] More 100 "/search/adult-site-keywords" Crawl errors under Search Console
-
I just opened my G Search Console and was shocked to see more than 150 Not Found errors under Crawl errors. Mine is a Wordpress site (it's consistently updated too):
Here's how they show up:
Example 1:
- URL: www.example.com/search/adult-site-keyword/page2.html/feed/rss2
- Linked From: http://an-adult-image-hosting.com/search/adult-site-keyword/page2.html
Example 2 (this surprised me the most when I looked at the linked from data):
-
URL: www.example.com/search/adult-site-keyword-2.html/page/3/
-
Linked From:
-
www.example.com/search/adult-site-keyword-2.html/page/2/ (this is showing as if it's from our own site)
-
http://a-spammy-adult-site.com/search/adult-site-keyword-2.html
Example 3:
- URL: www.example.com/search/adult-site-keyword-3.html
- Linked From: http://an-adult-image-hosting.com/search/adult-site-keyword-3.html
How do I address this issue?
-
Here is what I would do
-
Disavow the domain that is linking to you from the adult site(s).
-
The fact that Google search console is showing that you have an internal page linking as well makes me want to know a) have you always owned this domain and maybe someone previously did link internally like this or b) you may have been or are hacked
In the case of b) this can be really tricky. I once had a site that in a crawl it was showing sitewide links to various external sites that we should not be linking to. When I looked at the internal pages via my browser, there was no link as far as I could see even though it showed up on the crawler report.
Here was the trick. The hacker had setup a script to only show the link when a bot was viewing the page. Plus, we were running mirrored servers and they had only hacked one server. So, the links only showed up when you were spidering a specific mirrored instance as a bot.
So thanks to the hacking, not only were we showing bad links to bad sites, we were doing this through cloaking methodology. Two strikes against us. Luckily we picked this up pretty quick and fixed immediately.
Use a spidering program or browser program to show a user agent of Googlebot and go visit your pages that are linking internally. You might be surprised.
Summary
Googlebot has a very long memory. It may be that this was an old issue that was fixed long ago. If that was the case, just show the 404s for the pages that do not exist, and disavow the bad domain and move on. Make sure that you have not been hacked as this would also be why this is showing.
Regardless, the fact that Google did find it at one point, you need to make sure you resolve. Pull all the URLs into a spreadsheet and run Screaming Frog in list mode to check them all to make sure you fix all of it.
-
-
Yep.. Looking if anyone can help with this..
-
Oh yea, I missed that. That's very strange, not sure how to explain that one!
-
Thanks for the response Logan. What you are saying definitely makes sense.. But it makes think why do I see something like Example 2 under Crawl errors. Why Google Search Console shows linked from as 2 URL - one the spammy site's and other is from my own website. How is that even possible?
-
I've seen similar situations, but never in bulk and not with adult sites. Basically what's happening is somehow a domain (or multiple) are linking to your site with inaccurate URLs. When bots crawling those sites find the links pointing to yours, they obviously hit a 404 page which triggers the error in Search Console.
Unfortunately, there's not too much you can do about this, as people (or automated spam programs) can create a link to any site and any time. You could disavow links from those sites, which might help from an SEO perspective, but it won't prevent the errors from showing up in your Crawl Error report.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is it a good idea to optimize for keywords that have no search volume if they're ranked?
Hello Moz Community, I have some questions I hope some of you can help with. We’re doing SEO work for a client that provides outsourced IT and managed IT services in Phoenix, AZ and cities in the Phoenix metro area (i.e. Glendale, Tempe, Scottsdale, etc.) They’re currently ranked for or targeting the following keywords: • consulting phoenix az (1)
Intermediate & Advanced SEO | | marnipatterson
• outsourced it phoenix (2)
• phoenix it support (3)
• it services Scottsdale (5)
• it consulting firm phoenix (targeting)
• it solutions phoenix (targeting) We have recommended the following keywords based on monthly search totals, competitive level and difficulty ratings in Moz. • IT consulting phoenix
• it consultant company
• outsourced it
• it support services
• it consulting services
• outsourcing it
• outsourced tech support Questions
1. While I know it’s a good idea to optimize for keywords that you're currently ranked for, there’s no search volume for any of these. So, I recommended non-geo versions since Google provides search results based on the user’s location. Will this preserve the company's current rankings?
2. If not optimizing for their current keywords will hurt their rankings, will using the current keywords as secondary keywords suffice? If so, do we need to include them in the content for keyword density?
3. Since search engine algorithms now focus so heavily on user intent, I assume we’re covered for all variations of a keyword (i.e. outsource it, outsourced it, outsourcing it, etc.) Is this correct?
4. They want to rank for “cloud services” and “cloud solutions.” Both are very competitive with high difficulty rankings. So, I recommended “cloud migration” and “cloud strategy” as alternatives since these are the main services they provide. Will including “cloud services” and “cloud solutions” as secondary keywords help them increase their rankings for both? If you’ve dealt with a similar situation, I'd appreciate your insight and advice. Thanks!0 -
How long should you leave keywords on your site related to your old brand?
We rebranded our organization 2 years ago. We decided to leave some of our archived content with our old name because it was already serving link juice to the website. How long should you leave keywords like that on your site related to an old brand?
Intermediate & Advanced SEO | | sdaily0 -
Search Console - Best practice to fetch pages when you update them?
Hi guys, If you make changes to a page e.g. add more content or something is it good practice to get google to fetch that page again in search console? My assumption is this way, Google can review the updated page quicker, resulting in faster changes in the SERPs for that page. Thoughts? Cheers.
Intermediate & Advanced SEO | | wozniak650 -
Only ranking well when "UK" is added to search term
Hi, what does it mean when a lot of our keyword phrases rank only when "UK" is typed in the search term? For example:
Intermediate & Advanced SEO | | Solid_Web
"boxes" (not in top 50)
"boxes UK" (38) "big storage boxes" (45)
"big storage boxes UK" (33) We haven't attempted to SEO the pages for search terms with "UK" appended to them. Our domain is a co.uk domain. So, what reasons could there be that are we ranking in such a way?0 -
Should Site Search results be blocked from search engines?
What are the advantages & disadvantages of letting Google crawl site search results? We currently have them blocked via robots.txt, so I'm not sure if we're missing out on potential traffic. Thanks!
Intermediate & Advanced SEO | | pbhatt0 -
"nocontent" class use for Google Custom Search: SEO Ramifications?
Hi all, Have a client that uses Google Custom Search tool which is crawling, indexing and returning millions of irrelevant results for keywords that are on every page of the site. IT/Web dev. team is considering adding a class attribute to prohibit Google Custom Search from indexing bolierplate content regions. Here's the link to Google's custom search help page: http://support.google.com/customsearch/bin/answer.py?hl=en&answer=2364585 "...If your pages have regions containing boilerplate content that's not relevant to the main content of the page, you can identify it using the nocontent class attribute. When Google Custom Search sees this tag, we'll ignore any keywords it contains and won't take them into account when calculating ranking for your Custom Search engine. (We'll still follow and crawl any links contained in the text marked nocontent.) To use the nocontent class attribute, include the boilerplate content in a tag (for example, span or div) like this: Google Custom Search also notes:"Using nocontent won't impact your site's performance in Google Web Search, or our crawling of your site, in any way. We'll continue to follow any links in tagged content; we just won't use keywords to calculate ranking for your Custom Search engine."Just want to confirm if anyone can forsee any SEO implications the use of this div could create? Anyone have experience with this?Thank you!
Intermediate & Advanced SEO | | MRM-McCANN0 -
Should I use the main keyword in the title tag for the site on all category pages?
I am pretty excited about changing all my title tags (for the most important 7 pages) since I have seen my rankings jump up in the SERP just by adding the main keyword for my website in the title tag. To make it easier I will explain my business. Simply, I run an online jewelry shop, so basically the keywords I want to use is "Jewelry online" and for the main categories "Necklace", "Rings" and "Bracelets". What I am unsure about is whether to use all the keywords in the main pages title tag or should I just use the main keyword "Jewelry online". I don’t want to create competition between my own pages of course. Jewelry Online - Trendy Fashion Jewelry | Homepage Or Jewelry Online - Necklace, Rings, Bracelets | Homepage And the same goes for the main categories, should I include "jewelry online" or not, like: Bracelets - Fashion Jewelry Online | Homepage Or Bracelets - Trendy_ Bangles_ and Arm Cuffs | Homepage Any suggestions what is the best practice for the title tag on main page and the main categories? Thanks
Intermediate & Advanced SEO | | ikomorin0 -
WWW vs Non-WWW/Moving a site to a new CMS/Redirect all of the previous URLs
We are working on a new design for a website, which is currently on a CMS that has non-seo-friendly URLs. There is no redirection of 'www' to non-www or vice versa, or handling of homepage redirection so there is only one instance of 'home'. To move the site in the future, all of these URLs will have to be redirected to their new, and I hope, seo-friendly counterparts. Is it prudent now to redirect the four home page links so there is only one? and to redirect all non-www to 'www' so there is only one instance of each page? Or should I leave it and redirect all of them when the time comes?
Intermediate & Advanced SEO | | haan_seo0