Google indexing site content that I did not wish to be indexed
-
Hi is it pretty standard for Google to index content that you have not specifically asked them to index i.e. provided them notification of a page's existence.
I have just been alerted by 'Mention' about some new content that they have discovered, the page is on our site yes and may be I should have set it to NO INDEX but the page only went up a couple of days ago and I was making it live so that someone could look at it and see how the page was going to look in its final iteration. Normally we go through the usual process of notifying Google via GWMT, adding it to our site map.xml file, publishing it via our G+ stream and so on.
Reviewing our Analytics it looks like there has been no traffic to this page yet and I know for a fact there are no links to this page. I am surprised at the speed of the indexation, is it a example of brand mention? Where an actual link is now no longer required?
Cheers
David
-
Thanks Candyman, yes this is not a question about to prevent Google for not indexing my content, I know this very well. It is more about how quick they have done this with the least amount of effort on our part to inform them.
Plus it is quite an interesting situation you found yourself in, never heard of this before.
Many thanks
David
-
Hi David-
We had a similar situation recently where we had a dev site and forgot to no-index it and actually started to appear in the SERPS. After a bit of puzzling it LOOKS like Google found (or at least indexed) the pages as a function of us being logged into our Google accounts when viewing them. We did not do extensive testing on this, its mostly anecdotal but ti did look like it was true. Maybe we'll do the experiment one day to be sure!
Ken
-
Google is constantly indexing and viewing your website. Why go through the other steps? To ensure that your new page isn't overlooked. While you don't necessarily need to tell Google to index in GWT - your site map should automatically update, and if referenced in the robots.txt file than the new page will be found without issue.
Now, again if you don't want a page indexed and it has links than you need to do the noindex / no follow on the page, as the robots.txt can be over-ruled.
-
Hi Samuel,
Thanks for replying but no I'm not asking that, this I know how to do. The question is about whether this could be seen as an example of page indexation where on my part there has been no explicit activity to inform Google of the content's existence and there are no links to it yet Google is still managing to index it. Why bother informing Google vIA some of the activities mentioned earlier when they will just index it anyway you know.
Thanks
David
-
Are you asking how to prevent certain pages from appearing in search results? If so, I'd review Moz's guide to robots.
Specifically, I'd recommend the use of both the noindex meta tag and the robots.txt file. Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site has disappeared since Panda 4 despite quality content, help!
Our site www.physicalwellbeing.co.uk has lost over 20 first page rankings since the end of May. I assume this is because of Panda 4.0. All content on the site is high quality and 100% unique, so we did not expect to get penalised. Although I read somewhere that if Google can't read particular js anymore they don't rank you as high. The site has not been blacklisted as all pages are showing in Google's index and there are no messages on webmaster tools. We have not taken part in any link schemes and have disavowed all low quality links that were pointing there just in case (after the penalty). Can anybody see anything on www.physicalwellbeing.co.uk that may have cause Panda update to affect it so negatively? Would really appreciate any help.
Algorithm Updates | | search_shop0 -
Does adding lots of new content on a site at one time actually hurt you?
When speaking with a client today, he made the comment that he didn't want all of the new content we'd been working to be added to the site all at once for fear that he would get penalized for flooding the site with new content. I don't have any strong data to confirm or refute the claim, is there any truth to it?
Algorithm Updates | | JordanRussell0 -
Google has indexed a lot of test pages/junk from the development days.
With hind site I understand that this could have been avoided if robots.txt was configured properly. My website is www.clearvisas.com, and is indexed with both the www subdomain and with out. When I run site:clearvisas.com in Google I get 1,330 - All junk from the development days. But when I run site:www.clearvisas.com in Google I get 66 - these results all post development and more in line with what I wanted to be indexed. Will 1,330 junk pages hurt my seo? Is it possible to de-index them and should I? If the answer is yes to any of the questions how should I proceed? Kind regards, Fuad
Algorithm Updates | | Fuad_YK0 -
Organizing Website Content
Hey everyone! I am looking for some opinions on organizing website content. Here are two thoughts. I am open to alternative suggestions as well. Thanks for any consideration in this matter! **Aviation Supply Store Thought A ** **Helicopters Airplanes Rockets ** Helicopter engines Airplane engines Rocket engines Helicopter Fuels Airplane Fuels Rocket fuels Helicopter Accessories Airplane Accessories Rocket Accessories Aviation Supply Store Thought B Engines Fuels Accessories Helicopter Engines Helicopter Fuels Helicopter Accessories Airplane Engines Airplane Fuels Airplane Accessories Rocket Engines Rocket Fuels Rocket Accessories I simply chose aviation as an example. I'm just having difficulty deciding on how best to catagorize. Thank You!
Algorithm Updates | | APICDA0 -
Need help with some duplicate content.
I have some duplicate content issues on my blog I'm trying to fix. I've read lots of different opinions online about the best way to correct it, but they all contradict each other. I was hoping I could ask this community and see what the consensus was. It looks like my category and page numbers are showing duplicate content. For instance when I run the report I see things like this: http://noahsdad.com/resources/ http://noahsdad.com/resources/page/2/ http://noahsdad.com/therapy/page/2/ I'm assuming that is just the categories that are being duplicated, since the page numbers only show on the report at the end of a category. What is the best way to correct this? I don't use tags at all on my blog, using categories instead. I also use the Yoast SEO plug in. I have a check mark in the box that disables tags. However it says, "If you're using categories as your only way of structure on your site, you would probably be better off when you prevent your tags from being indexed." There is a box that allows you to disable categories also, but the description above makes it seem like I don't want to block both tags and categories. Any ideas what I should do? Thanks.
Algorithm Updates | | NoahsDad0 -
Are multiple domains for my website hurting my Google ranking?
Hello, I currently have two domains showing up in google search: shwoodshop.com shop.shwoodshop.com These domains are currently ranked in the #2 and #3 spot, however my page is much more trafficked than the current #1 ranking. I am wondering if the fact that I have two domains competing for the #1 spot is hurting my search ranking. If so, what is the best way to remedy this issue and get back my #1 spot? I'm rather new to SEO and teaching myself as I go, so I appreciate the feedback!
Algorithm Updates | | shwoodshop0 -
Organic CTR on Google - KPI?
Hi, I was hoping for some advice on my keyword analysis I have completed. So far I have identified a hitlist of high volume keyword associated to the industry I operate in. As well as this, I'm monitoring our keyword positions within the SERPS. Question: Is there a CTR metric available depending on the position your keyword ranks within Google? i.e. If I am position 3 and looking to move to position 1 on a specific keyword, what amount of incremental search volume would be geneerated to my website? PResumably the CTR would also depend on what market you operate in too I am also going on a 65% / 35% Organic/PPC split based on keyword search volume so to give me a true reflection of the search volume available... Any advice on this would be much appreciated... Simon
Algorithm Updates | | simonsw0 -
Index Page lost rankings? Please Help!
This morning I ranked highly (Page 1 UK Google) for over 50 keyword search terms for my website http://www.careworx.co.uk This afternoon my rankings have bottomed out and dropped pages? I have not been de-indexed it appears and many of my sub-pages are still highly ranked. Would anybody know what has happened? I know of Google Panda but I would've seen results drop before now so I'm very concerned. Don't seem to have lost any links etc and am careful to balance SEO with a mix of techniques to keep Google happy and again, have not been de-indexed. Can anybody offer advice please, or let me know how I can rectify this.
Algorithm Updates | | andystep0