URLs dropping from index (Crawled, currently not indexed)
-
I've noticed that some of our URLs have recently dropped completely out of Google's index.
When carrying out a URL inspection in GSC, it comes up with 'Crawled, currently not indexed'.
Strangely, I've also noticed that under referring page it says 'None detected', which is definitely not the case.
I wonder if it could be something to do with the following? https://www.seroundtable.com/google-ranking-index-drop-30192.html - It seems to be a bug affecting quite a few people.
Here are a few examples of the URLs that have gone missing:
https://www.ihasco.co.uk/courses/detail/sexual-harassment-awareness-training
https://www.ihasco.co.uk/courses/detail/conflict-resolution-training
https://www.ihasco.co.uk/courses/detail/prevent-duty-training
Any help here would be massively appreciated!
-
@Philljones22 said in URLs dropping from index (Crawled, currently not indexed):
Same issue here, my most of the URLs are getting de indexed after indexing the search console.
I'm experiencing the same problem. Most of my URLs are getting de-indexed after being indexed by the search console.
https://www.stardewvalleyapk.me/
https://www.stardewvalleyapk.me/stardew-valley-mod-apk/ -
I don't know why but I am facing the same issue from past 3 monts.
My most of the URLs are getting de indexed after indexing the search console. -
@kingshah001 said in URLs dropping from index (Crawled, currently not indexed):
Same issue here, my most of the URLs are getting de indexed after indexing the search console.
Same issue here, my most of the URLs are getting de indexed after indexing the search console.
https://inshotproapps.com
https://instoproapps.com/inshot-for-pc/ -
Thanks for sharing details.
-
Same issue here, my most of the URLs are getting de indexed after indexing the search console.
Some of the URLs are below:
https://apkcroc.com/
https://apkcroc.com/vn-mod-apk/
https://apkcroc.com/kinemaster-mod-apk/
https://apkcroc.com/terragenesis-mod-apk/
https://apkcroc.com/sky-fighters-3d-mod-apk/ -
My site is also a victim of same the issue, collecting bits and actionable advice. I'm planning to post my experience on Moz forum soon.
-
Hello,
since the beginning of ladykiller.nl I am having the same issues with Google to crawl sitemap(s) and index urls. I am using Yoast as a plugin for the sitemap.
For the moment +3620 urls are indexed, but my website has +10.000 urls :(.
Also from time to time I get a notice in GSC that Google can not fetch certain sitemap urls f.e. https://ladykiller.nl/post-sitemap.xml. Mostly the issue is fixed after a week or so. Please find print screen here: https://prnt.sc/Pm_h2Arjxu-kAlready asked on numerous forums for help, as I can not find a solution to get this problem fixed. However, without any good results so far.
Therefore, I am trying it here again in the hope maybe some of you guys have some better understanding of what the issue might be and how it can be fixed. All help is highly appreciated!
Thanks in advance for having a look into it :)!
Warm regards,
John -
Hi there,
The third URL you are referencing, is actually indexed:
https://dmitrii-regexseo.tinytake.com/tt/NDY4NDY4N18xNDgzNjgzMA
As for "crawled, not indexed" - in most cases it happens because of one and only reason - Google is seeing your page as thin content, not worth being indexed. Typically it happens on bigger sites with a lot of similar pages. In your case, you got many courses, with exactly same structure. So, if the content is not completely different, then Google might deem it not worthy.
As for the bug you referenced - did your URLs drop off the index exactly at the time when this issue has been discovered? (aka within the last week?).
Do you have any cannibalization happening?
To me it looks like that's the case. If I do this search: "site:https://www.ihasco.co.uk/ Sexual Harassment Training course"
There are many pages that are indexed and are ranking: https://dmitrii-regexseo.tinytake.com/tt/NDY4NDcwN18xNDgzNjg4Mg
So, basically, you have pages that are more authoritative with similar content. Therefore your courses pages are dropping as thin content.
I would recommend doing some internal linking optimization to tell Google what is actually important. Look in GSC for internal links metrics.
Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL Parameters as pagination
Hi guys, due to some changes to our category pages our paginated urls will change so they will look like this: ...category/bagger/2?q=Bagger&startDate=26.06.2017&endDate=27.06.2017 You see they include a query parameter as well as a start and end date which will change daily. All URLs with pagination are on noindex/follow. I am worrying that the products which are linked from the category pages will not get crawled well when the URLs on which they are linked from change on a daily basis. Do you have some experience with this? Are there other things we need to worry about with these pagination URLs? cheers
Technical SEO | | JKMarketing0 -
Index bloating issue
Hello, In the last month, I noticed a huge spike in the number of pages indexed on my site, which I think is impacting my SEO quality score. While I've only have about 90 pages on my site map, the number of pages indexed jumped to 446, with about 536 pages being blocked by robots. At first we thought this might be due to duplicate product pages showing up in different categories on my site, but we added something to our robot.txt file to not index those pages. But the number has not gone down. I've tried to consult with our hosting vendor, but no one seems to be concerned or have any idea why there was such a big jump in the last month. Any insights or pointers would be so greatly appreciated, so that I can fix/improve my SEO as quickly as possible! Thanks!
Technical SEO | | Saison0 -
My website pages are not crawled, what to do?
Hi all. I have made some changes on the website so i like to crawled them by the search engines Google especially. I have made these changes around 2 weeks ago. I have submitted my website on good bookmarking websites. Also i used a tool available in Google webmasters "Fetch as Google", Resubmitted a sitemap.xml. Still my pages are not crawled your opinion please. Thanks
Technical SEO | | lucidsoftech0 -
Odd URL errors upon crawl
Hi, I see this in Google Webmasters, and am now also seeing it here...when a crawl is performed on my site, I get many 500 server error codes for URLs that I don't believe exist. It's as if it sees a normal URL but adds this to it: %3Cdiv%20id= It's like this for hundreds of URLs. Good URL that actually exists http://www.ffr-dsi.com/food-retailing/supplies/ URL that causes error and I have no idea why http://www.ffr-dsi.com/food-retailing/supplies/%3Cdiv%20id= Thanks!
Technical SEO | | Matt10 -
Updating content on URL or new URL
High Mozzers, We are an event organisation. Every year we produce like 350 events. All the events are on our website. A lot of these events are held every year. So i have an URL like www.domainname.nl/eventname So what would you do. This URL has some inbound links, some social mentions and so on. SO if the event will be held again in 2013. Would it be better to update the content on this URL or create a new one. I would keep this URL and update it because of the linkvalue and it is allready indexed and ranking for the desired keyword for that event. Cheers, Ruud
Technical SEO | | RuudHeijnen0 -
Index page
To the SEO experts, this may well seem a silly question, so I apologies in advance as I try not to ask questions that I probably know the answer for already, but clarity is my goal I have numerous sites ,as standard practice, through the .htaccess I will always set up non www to www, and redirect the index page to www.mysite.com. All straight forward, have never questioned this practice, always been advised its the ebst practice to avoid duplicate content. Now, today, I was looking at a CMS service for a customer for their website, the website is already built and its a static website, so the CMS integration was going to mean a full rewrite of the website. Speaking to a friend on another forum, he told me about a service called simple CMS, had a look, looks perfect for the customer ... Went to set it up on the clients site and here is the problem. For the CMS software to work, it MUST access the index page, because my index page is redirected to www.mysite.com , it wont work as it cant find the index page (obviously) I questioned this with the software company, they inform me that it must access the index page, I have explained that it wont be able to and why (cause I have my index page redirected to avoid duplicate content) To my astonishment, the person there told me that duplicate content is a huge no no with Google (that's not the astonishing part) but its not relevant to the index and non index page of a website. This goes against everything I thought I knew ... The person also reassured me that they have worked within the SEO area for 10 years. As I am a subscriber to SEO MOZ and no one here has anything to gain but offering advice, is this true ? Will it not be an issue for duplicate content to show both a index page and non index page ?, will search engines not view this as duplicate content ? Or is this SEO expert talking bull, which I suspect, but cannot be sure. Any advice would be greatly appreciated, it would make my life a lot easier for the customer to use this CMS software, but I would do it at the risk of tarnishing the work they and I have done on their ranking status Many thanks in advance John
Technical SEO | | Johnny4B0 -
Shorter URLs
Hi Is there a real value in having the keywords in the URL structure? we could use the URL: Mybrand.com/software/tablets/ipad/supertrader.html Or instead have the CMS create the shorter version mybrand.com/supertrader.html and just optimize this page for the keyword 'supertrader ipad software'
Technical SEO | | FXDD1 -
IP address URLs being indexed, 301 to domain?
I apoligize if this question as been asked before, I couldnt' find in the Q&A though. I noticed Google has been indexing our IP address for some pages (ie: 123.123.123.123/page.html instead of domain.com/page.html). I suspect this is possibly due to a few straggler relative links instead of absolute, or possibly something else I'm not thinking of. My less-evasive solution a few months back was to ensure canonical tags were on all pages, and then replaced any relative links w/ absolutes. This does not seem to be fixing the problem though, as recently as today new pages were scooped up with the IP address. My next thought is to 301 redirect any IP address URL to the domain, but I was afraid that may be too drastic and that the canonical should be sufficient (which it doesn't seem to be). Has anyone dealt with this issue? Do you think the 301 would be a safe move, any other suggestions? thanks.
Technical SEO | | KT6840