Google indexing site content that I did not wish to be indexed
-
Hi is it pretty standard for Google to index content that you have not specifically asked them to index i.e. provided them notification of a page's existence.
I have just been alerted by 'Mention' about some new content that they have discovered, the page is on our site yes and may be I should have set it to NO INDEX but the page only went up a couple of days ago and I was making it live so that someone could look at it and see how the page was going to look in its final iteration. Normally we go through the usual process of notifying Google via GWMT, adding it to our site map.xml file, publishing it via our G+ stream and so on.
Reviewing our Analytics it looks like there has been no traffic to this page yet and I know for a fact there are no links to this page. I am surprised at the speed of the indexation, is it a example of brand mention? Where an actual link is now no longer required?
Cheers
David
-
Thanks Candyman, yes this is not a question about to prevent Google for not indexing my content, I know this very well. It is more about how quick they have done this with the least amount of effort on our part to inform them.
Plus it is quite an interesting situation you found yourself in, never heard of this before.
Many thanks
David
-
Hi David-
We had a similar situation recently where we had a dev site and forgot to no-index it and actually started to appear in the SERPS. After a bit of puzzling it LOOKS like Google found (or at least indexed) the pages as a function of us being logged into our Google accounts when viewing them. We did not do extensive testing on this, its mostly anecdotal but ti did look like it was true. Maybe we'll do the experiment one day to be sure!
Ken
-
Google is constantly indexing and viewing your website. Why go through the other steps? To ensure that your new page isn't overlooked. While you don't necessarily need to tell Google to index in GWT - your site map should automatically update, and if referenced in the robots.txt file than the new page will be found without issue.
Now, again if you don't want a page indexed and it has links than you need to do the noindex / no follow on the page, as the robots.txt can be over-ruled.
-
Hi Samuel,
Thanks for replying but no I'm not asking that, this I know how to do. The question is about whether this could be seen as an example of page indexation where on my part there has been no explicit activity to inform Google of the content's existence and there are no links to it yet Google is still managing to index it. Why bother informing Google vIA some of the activities mentioned earlier when they will just index it anyway you know.
Thanks
David
-
Are you asking how to prevent certain pages from appearing in search results? If so, I'd review Moz's guide to robots.
Specifically, I'd recommend the use of both the noindex meta tag and the robots.txt file. Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My site is showing indexed in search console but not appearing in Serps
hi, i have recently made sites.google site and submitted to search console but when I copy paste in google , its not appearing
Algorithm Updates | | alan-shultis0 -
Google Cache
So, when I gain a link I always check to see if the page that is linking is in the Google cache. I've noticed recently that more and more pages are actually not showing up in Google's cache, yet still appear in search results. I did read an article from someone whoo works at Google a few weeks back that there is sometimes an error with the cache and occasionally the cache will not display. This week, my own website isn't showing up in the cache yet I'm still ranking in SERP's. I'm not worried about it, mostly whitehat, but has there been any indication that Google are phasing out the ability to check cache's of websites?
Algorithm Updates | | ThorUK0 -
Duplicate website pages indexed: Ranking dropped. Does Google checks the duplicate domain association?
Hi all, Our duplicate website which is used for testing new optimisations got indexed and we dropped in rankings. But I am not sure whether this is exact reason as it happened earlier too where I don't find much drop in rankings. Also I got replies in the past that it'll not really impact original website but duplicate website. I think this rule applies to the third party websites. But if our own domain has exact duplicate content; will Google knows that we own the website from any other way we are associated like IP addresses and servers, etc..to find the duplicate website is hosted by us? I wonder how Google treats duplicate content from third party domains and own domains. Thanks
Algorithm Updates | | vtmoz0 -
Google is forcing a 301 by truncating our URLs
Just recently we noticed that google has indexed truncated urls for many of our pages that get 301'd to the correct page. For example, we have:
Algorithm Updates | | mmac
http://www.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html as the url linked everywhere and that's the only version of that page that we use. Google somehow figured out that it would still go to the right place via 301 if they removed the html filename from the end, so they indexed just: http://www.eventective.com/USA/Massachusetts/Bedford/107/ The 301 is not new. It used to 404, but (probably 5 years ago) we saw a few links come in with the html file missing on similar urls so we decided to 301 them instead thinking it would be helpful. We've preferred the longer version because it has the name in it and users that pay attention to the url can feel more confident they are going to the right place. We've always used the full (longer) url and google used to index them all that way, but just recently we noticed about 1/2 of our urls have been converted to the shorter version in the SERPs. These shortened urls take the user to the right page via 301, so it isn't a case of the user landing in the wrong place, but over 100,000 301s may not be so good. You can look at: site:www.eventective.com/usa/massachusetts/bedford/ and you'll noticed all of the urls to businesses at the top of the listings go to the truncated version, but toward the bottom they have the full url. Can you explain to me why google would index a page that is 301'd to the right page and has been for years? I have a lot of thoughts on why they would do this and even more ideas on how we could build our urls better, but I'd really like to hear from some people that aren't quite as close to it as I am. One small detail that shouldn't affect this, but I'll mention it anyway, is that we have a mobile site with the same url pattern. http://m.eventective.com/USA/Massachusetts/Bedford/107/Doubletree-Hotel-Boston-Bedford-Glen.html We did not have the proper 301 in place on the m. site until the end of last week. I'm pretty sure it will be asked, so I'll also mention we have the rel=alternate/canonical set up between the www and m sites. I'm also interested in any thoughts on how this may affect rankings since we seem to have been hit by something toward the end of last week. Don't hesitate to mention anything else you see that may have triggered whatever may have hit us. Thank you,
Michael0 -
Phantom Indexed: 301 Redirected Old URL Shows in Google Search Result!
Today, I have read about Phantom Indexed in Google search result. Because, I was searching about 301 redirect due to indexing of 301 redirected old URLs in Google search result rather than new landing pages. I've added my comment on jennita's blog post about 301 redirect. I would like to paste similar question over here! I have 301 redirected following 3 domains to new website... http://www.lampslightingandmore.com/ To http://www.vistastores.com/table-lamps http://www.vistapatioumbrellas.com/ To http://www.vistastores.com/patio-umbrellas http://www.spiderofficechairs.com/ To http://www.vistastores.com/office-chairs I have done it before 3 months but, Google still shows me home page URL in search result rather than new landing page. You can check following search results to know more about it. For LampsLightingandMore ~ On second or third page::: For VistaPatioUmbrellas ~ On second or third page::: For SpiderOfficeChairs ~ On Second or third page::: I come to know about Phantom Indexed after raised my comment over there. So, why should not start discussion on it. Because, It's all about branding and who'll love to hang old address in front of new home.
Algorithm Updates | | CommercePundit0 -
When Google crawls and indexes a new page does it show up immediately in Google search - "site;"?
We made changes to a site, including the addition of a new page and corresponding link/text changes to existing pages. The changes are not yet showing up in the Google index (“site:”/cache), but, approximately 24 hours after making the changes, The SERP's for this site jumped up. We obtained a new back link about a couple of weeks ago, but it is not yet showing up in OSE, Webmaster Tools, or other tools. Just wondering if you think the Google SERP changes run ahead of what they actually show us in site: or cache updates. Has Google made a significant SERP “adjustment” recently? Thanks.
Algorithm Updates | | richpalpine0 -
Google new update question
I was just reading this, http://www.entrepreneur.com/blog/220662 We have our official site, which has 200+ service pages, which we wrote once and we keep doing SEO for them, so they rank high all the time. Now my question is, how does Google handle the site freshness ? Service static pages or if we are adding blog items, then also they consider them as fresh site, right ? So, we dont have to update those service pages, right ?
Algorithm Updates | | qubesys0 -
Do we have a timeline of google, bing updates
I thought it would be handy if we had a timeline with dates of any updates to the algo's.
Algorithm Updates | | AlanMosley
Does one exists here at SEOMoz or elsewhere.
Thanks3