If a page isn't linked to or directly sumitted to a search engine can it get indexed?
-
Hey Guys,
I'm curious if there are ways a page can get indexed even if the page isn't linked to or hasn't been submitted to a search engine.
To my knowledge the following page on our website is not linked to and we definitely didn't submit it to Google - but it's currently indexed:
<cite>takelessons.com/admin.php/adminJobPosition/corp</cite>
Anyone have any ideas as to why or how this could have happened? Hopefully I'm missing something obvious
Thanks,
Jon
-
You're welcome Jon.
That's a good question. I don't know the official answer on that one, though suspect that Google does check to see if the page exists, mainly because often it will be a valid URL that somebody types in the search box instead of the address bar. If Google don't have that page in their index, they'd at least like to consider adding it.
http://www.google.com/addurl.html is the URL adding page for Google as you'll already know. As well as Google relying on people to use this form, they will also, I suspect, crawl URLs that are entered into a search box. Makes sense to me that Google would at least visit these pages searched on, though can't be sure.
Regards
Simon
-
Thanks for the responses guys!
Are either of you aware of whether or not Google would ever index a page if someone searched for it? For example, say someone did a Google search for the URL I specified above. Would Google ever get curious and then try and crawl it?
Thanks again for taking the time to help out
-
Hey there Jon.
It seems that it was last indexed almost a month ago on Oct. 21, 2011. I would suggest that you follow Simon's advice on the NoFollow tag.
Additionally, check your GWT to ensure that it is not indexed...If it is, then try to resubmit it to get indexed (I know, I know, it doesn't make sense), but it will send out a message to crawl it. NoFollow, NoIndex tells the spider NOT HERE....Anywho, good luck with that and let us know how it turned out!
Cheers!
P.S. At least it's not indexed in Bing
-
Hi Jon
This is a strange one, I too haven't found a link to your admin page. There could have been one at some point in the past. Google's bot is rather clever at finding pages on the so-called 'invisible web', so best to request non-indexing of pages that you don't want indexed (covered below).
You're right in thinking that search bots follow links to find and index pages, whether it be an external or internal site link or a sitemap link. They also find pages through actual submissions.
- I'd suggest modifying the Robots tag on your admin page, include a 'NoIndex' just before the NoFollow'.
- Also include a Disallow command in your Robots.txt file for this admin page, and perhaps all pages within the admin section.
- Then, request the URL be removed from Google's index via Google Webmaster Tools ("Site configuration", "Crawler access", "Remove URL").
Hope that helps,
Regards
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can't get my site recognised for keyword
My site prettycool.co.uk and primary we sell fascinators, the problem is I can't get the word fascinators to be listed by Google. We are on the 1st page for most colours ie. pink fascinators, blue fascinators etc. but for the term fascinators even if we fetch we are listed for a couple of hours and then disappear. I've checked for keyword stuffing but our site sell fascinators and we need to have this word in our site and other sites have a lot more references to the term and are listed on the 1st or 2nd pages. We used to be listed on page 1 for many years but the last 2 or 3 years dropped back to page 4 but now nothing. Any help or suggestions would be fantastic!
Technical SEO | | Rutts0 -
Indexing Issue of Dynamic Pages
Hi All, I have a query for which i am struggling to find out the answer. I unable to retrieve URL using "site:" query on Google SERP. However, when i enter the direct URL or with "info:" query then a snippet appears. I am not able to understand why google is not showing URL with "site:" query. Whether the page is indexed or not? Or it's soon going to be deindexed. Secondly, I would like to mention that this is a dynamic URL. The index file which we are using to generate this URL is not available to Google Bot. For instance, There are two different URL's. http://www.abc.com/browse/ --- It's a parent page.
Technical SEO | | SameerBhatia
http://www.abc.com/browse/?q=123 --- This is the URL, generated at run time using browse index file. Google unable to crawl index file of browse page as it is unable to run independently until some value will get passed in the parameter and is not indexed by Google. Earlier the dynamic URL's were indexed and was showing up in Google for "site:" query but now it is not showing up. Can anyone help me what is happening here? Please advise. Thanks0 -
Site address change: new site isn't showing up in Google, old site is gone.
We just transitioned mccacompanies.com to confluentstrategies.com. The problem is that when I search for the old name, the old website doesn't come up anymore to redirect people to the new site. On the local card, Google has even taken off the website altogether. (I'm currently still trying to gain access to manage the business listing) When I search for confluent strategies, the website doesn't come up at all. But if I use the site: operator, it is in the index. Basically, my client has effectively disappeared off the face of the Google. (In doing other name changes, this has never happened to me before) What can I do?
Technical SEO | | MichaelGregory0 -
Should i Disavow links from Secret Search Engine Labs
Hi, I'm doing a link audit. My sites' keyword rankings and organic traffic have been sent to the Phantom Zone since the last Penguin Update. I've got 70+ and counting follow backlinks to my main domain and one of the my subdomains from http://www.secretsearchenginelabs.com/; should i disavow them? There's a load of links to many recognisable sites in there and my instinct's all out of whack with this decision. I think that these links were all manually added by a link building company on our behalf. It does look manipulated to me but i'd like a second opinion before I dump all of those links. Thanks Thanks
Technical SEO | | McCaldin0 -
So many internal links to the same page
Hey guyz,
Technical SEO | | atakala
I'm working with a client that has a page which has many internal links to the same page .
Let me illustrate it.
So as you can see I have a page which is called in the image "page" :D.
As you can see, the **page **has many links to the solutions.htmls' anchor links which mean they are basically the same page ( solutions.html)
Is it going to be a problem for us to do that ?
And is there anyway to handle this problem?
Thank you for you patience. And sorry for my bad english 😄 4deRc1W.png0 -
Why Custom Post Types Don't Get Ranked Well
So I have started to use custom post types and have noticed that they don't get ranked very well in Google. On the other hand a post will rank almost immediately. Does anyone know any tricks for making custom post types rank as well as posts in wordpress? Sincerely, Garret
Technical SEO | | eWebify0 -
Can you noindex a page, but still index an image on that page?
If a blog is centered around visual images, and we have specific pages with high quality content that we plan to index and drive our traffic, but we have many pages with our images...what is the best way to go about getting these images indexed? We want to noindex all the pages with just images because they are thin content... Can you noindex,follow a page, but still index the images on that page? Please explain how to go about this concept.....
Technical SEO | | WebServiceConsulting.com0 -
Should I delete a page or remove links on a penalized page?
Hello All, If I have a internal page that has low quality links point to it or a penality. Can I just remove the page, and start over versus trying to remove the links? Over time wouldn't this page disapear along with the penalty on that page? Kinda like pruning a tree? Cutting off the junk limbs so other could grow stronger, or to start new fresh ones. Example: www.domain.com Penalized Internal Page: (Say this page is penalized due to keyword stuffing, and has low quality links pointing to it like blog comments, or profiles) www.domain.com/penalized-internal-page.com Would it be effective to just delete this page (www.domain.com/penalized-internal-page.com) and start over with a new page. New Internal Page: www.domain.com/new-internal-page.com I would of course lose any good links point to that page, but it might be easier then trying to remove old back links. Thoughts? Thanks! Pete
Technical SEO | | Juratovic0