I'm noticing that URL that were once indexed by Google are suddenly getting dropped without any error messages in Webmasters Tools, has anyone seen issues like this before?
-
I'm noticing that URLs that were once indexed by Google are suddenly getting dropped without any error messages in Webmasters Tools, has anyone seen issues like this before?
Here's an example:
http://www.thefader.com/2017/01/11/the-carter-documentary-lil-wayne-black-lives-matter -
I tried your exact search and it's still not coming up for me.
I also used the GWT fetching and rendering tool on some other links that were once indexed and then removed- and what I noticed is that they would get reindexed for short time, and then will get removed from the index again.
We are submitting a sitemap through GWT and no errors are getting returned.
We must be having an issue with Google crawling our site, but I haven't been able to find anyone who has had a similar experience- do you know of any other things I can try to fix this issue. The main problem is that it's intermittent and there is no consistency as to why some of our links are indexed and some aren't.
-
With the site search, are you including "site:" in the search query before the URL, like in my example? You shouldn't be getting this thread in a site search if you're doing it right.
If the pages are indexed, then you just need to get them ranking in a better position. In which case, you should form an SEO strategy.
For sites that are no longer indexed, try fetching and rendering the URL in Google's Search Console to see if there's any issue. You can also request Google submit the URL to index through this tool.
If you haven't already, it would also be beneficial for you to submit your sitemap through Google's Search Console to improve Google's crawling of your site. It might just be that Google is having a hard time crawling your site, and that's why you're dropping links.
-
Neither one is showing as indexed for me, actually this messageboard thread is what I'm getting in my SERP.
I am noticing this randomly with a lot of our URLs, with no error messages in Webmasters Tools, any suggestions on how to fix this?
-
If you do a site search for the URL (just search in Google: "site:http://www.thefader.com/2017/01/09/lil-wayne-best-rapper-alive-rap-millennial-grows-up-essay" - without quotation marks), you can see if it's indexed in Google.
The first URL you gave was indexed for me. But the second one doesn't appear to be.
-
Hi Ria,
How are you determining that this URL has been indexed? When I search our page title, keywords that should return this result, or even the direct URL- this page doesn't show in any of the search results.
Here's another example that doesn't seem to be indexed:
http://www.thefader.com/2017/01/09/lil-wayne-best-rapper-alive-rap-millennial-grows-up-essayAm I missing something obvious?
-
I'm seeing that URL as being indexed in Google. Do you have any other examples?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How necessary is it to disavow links in 2017? Doesn't Google's algorithm take care of determining what it will count or not?
Hi All, So this is a obvious question now. We can see sudden fall or rise of rankings; heavy fluctuations. New backlinks are contributing enough. Google claims it'll take care of any low quality backlinks without passing pagerank to website. Other end we can many scenarios where websites improved ranking and out of penalty using disavow tool. Google's statement and Disavow tool, both are opposite concepts. So when some unknown low quality backlinks are pointing and been increasing to a website? What's the ideal measure to be taken?
Intermediate & Advanced SEO | | vtmoz0 -
Client wants to remove mobile URLs from their sitemap to avoid indexing issues. However this will require SEVERAL billing hours. Is having both mobile/desktop URLs in a sitemap really that detrimental to search indexing?
We had an enterprise client ask to remove mobile URLs from their sitemaps. For their website both desktop & mobile URLs are combined into one sitemap. Their website has a mobile template (not a responsive website) and is configured properly via Google's "separate URL" guidelines. Our client is referencing a statement made from John Mueller that having both mobile & desktop sitemaps can be problematic for indexing. Here is the article https://www.seroundtable.com/google-mobile-sitemaps-20137.html
Intermediate & Advanced SEO | | RosemaryB
We would be happy to remove the mobile URLs from their sitemap. However this will unfortunately take several billing hours for our development team to implement and QA. This will end up costing our client a great deal of money when the task is completed. Is it worth it to remove the mobile URLs from their main website to be in adherence to John Mueller's advice? We don't believe these extra mobile URLs are harming their search indexing. However we can't find any sources to explain otherwise. Any advice would be appreciated. Thx.0 -
Website dropped out from Google index
Howdy, fellow mozzers. I got approached by my friend - their website is https://www.hauteheadquarters.com She is saying that they dropped from google index over night - and, as you can see if you google their name, website url or even site: , most of the pages are not indexed. Home page is nowhere to be found - that's for sure. I know that they were indexed before. Google webmaster tools don't have any manual actions (at least yet). No sudden changes in content or backlink profile. robots.txt has some weird rule - disallow everything for EtaoSpider. I don't know if google would listen to that - robots checker in GWT says it's all good. Any ideas why that happen? Any ideas what I should check? P.S. Just noticed in GWT there was a huge drop in indexed pages within first week of August. Still no idea why though. P.P.S. Just noticed that there is noindex x-robots-tag in headers... Anyone knows where this can be set?
Intermediate & Advanced SEO | | DmitriiK0 -
Difference in Number of URLS in "Crawl, Sitemaps" & "Index Status" in Webmaster Tools, NORMAL?
Greetings MOZ Community: Webmaster Tools under "Index Status" shows 850 URLs indexed for our website (www.nyc-officespace-leader.com). The number of URLs indexed jumped by around 175 around June 10th, shortly after we launched a new version of our website. No new URLs were added to the site upgrade. Under Webmaster Tools under "Crawl, Site maps", it shows 637 pages submitted and 599 indexed. Prior to June 6th there was not a significant difference in the number of pages shown between the "Index Status" and "Crawl. Site Maps". Now there is a differential of 175. The 850 URLs in "Index Status" is equal to the number of URLs in the MOZ domain crawl report I ran yesterday. Since this differential developed, ranking has declined sharply. Perhaps I am hit by the new version of Panda, but Google indexing junk pages (if that is in fact happening) could have something to do with it. Is this differential between the number of URLs shown in "Index Status" and "Crawl, Sitemaps" normal? I am attaching Images of the two screens from Webmaster Tools as well as the MOZ crawl to illustrate what has occurred. My developer seems stumped by this. He has submitted a removal request for the 175 URLs to Google, but they remain in the index. Any suggestions? Thanks,
Intermediate & Advanced SEO | | Kingalan1
Alan0 -
Google Semantic Search: Now I'm really confused
I'm struggling to understand why I rank for some terms and not for other closely related ones. For example: property in Toytown but NOT properties in toytown property for sale in Toytown but NOT property for sale Toytown NOR properties for sale Toytown. My gut instinct is that I don't have enough of the second phrasing as inbound link anchor text -- but didn't Penguin/Panda make all that obsolete?
Intermediate & Advanced SEO | | Jeepster0 -
Will implementing a 'Scroll to Div Anchor' cause a duplicate content issue?
I have just been building a website for a client with pages that contain a lot of text content. To make things easier for site visitors I have created a menu bar that sticks to the top of the page and the page will scroll to different areas of content (i/e different Div id anchors) Having done this I have just had the thought that this might inadvertently introduce duplicate content issue. Does anyone know if adding an #anchor to the end of a url will cause a duplicate content error in google? For example, would the following URLs be treated as different:- http://www.mysite.co.uk/services
Intermediate & Advanced SEO | | AdeLewis
http://www.mysite.co.uk/services#anchor1
http://www.mysite.co.uk/services#anchor2
http://www.mysite.co.uk/services#anchor3
http://www.mysite.co.uk/services#anchor4 Thanks.0 -
Every seen any SERP like this?
I've never seen a SERP like this, has anyone else? Image no longer available
Intermediate & Advanced SEO | | ATShock1 -
We are changing ?page= dynamic url's to /page/ static urls. Will this hurt the progress we have made with the pages using dynamic addresses?
Question about changing url from dynamic to static to improve SEO but concern about hurting progress made so far.
Intermediate & Advanced SEO | | h3counsel0