Can Search engines crawl this page
-
Hi guys,
To put a long story short we have had to make a copy of our site and put it on another domain so in essence there are 2 copies of our site on the web.What we have done is put a username and password on the homepage - http://www.ughhwiki.co.uk/ now i just want to be 100% sure that the search engines cannot crawl this?
Thank you
Jon
-
Google should not be able to crawl that page as long as it is password protected.
However, Google already has it indexed and password protecting it may not induce Google to de-index the page.
-
Hi Lewis,
Thanks for the reply. We are trying to prevent this site from being able to be crawled by anyone, search engines and visitors. The login page is hopefully able to prevent google from seeing any of our content while we edit it and then put it on our actual live url. Does that make sense?
-
Hi Jon
I wasn't able to access the page. Is the password just on the homepage though? If I was you I would test some other pages as you wouldn't want some one coming into the site via a search engine result if you are trying to keep everyone locked out at the moment.
Hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Replication on Search
Hi. We recently created a Christmas category page on our eCommerce website (christowhome.co.uk). Earlier today, I Googled ‘Christow Christmas Silhouette Lights’ (Christow being the name of our website and Christmas silhouette lights being one of the sub-categories we recently created). I was curious to see how the page appeared on search. Bizarrely, the page appeared multiple times on search (if you click on the link above, it should show you the search results). As you can see, multiple meta titles and descriptions have been created for the same page. This is something that is affecting a number of our Christmas category pages. I don't quite understand why this has happened. We recently added filters to the category. Could the filters be responsible? Any idea how I can prevent this from happening? How I can stop Google indexing these weird replica pages? Many thanks, Dave
Technical SEO | | Davden0 -
3,511 Pages Indexed and 3,331 Pages Blocked by Robots
Morning, So I checked our site's index status on WMT, and I'm being told that Google is indexing 3,511 pages and the robots are blocking 3,331. This seems slightly odd as we're only disallowing 24 pages on the robots.txt file. In light of this, I have the following queries: Do these figures mean that Google is indexing 3,511 pages and blocking 3,331 other pages? Or does it mean that it's blocking 3,331 pages of the 3,511 indexed? As there are only 24 URLs being disallowed on robots.text, why are 3,331 pages being blocked? Will these be variations of the URLs we've submitted? Currently, we don't have a sitemap. I know, I know, it's pretty unforgivable but the old one didn't really work and the developers are working on the new one. Once submitted, will this help? I think I know the answer to this, but is there any way to ascertain which pages are being blocked? Thanks in advance! Lewis
Technical SEO | | PeaSoupDigital0 -
Best Practice for Blocking a site from 1 countries search engines
A client cannot appear in any search engines in one given country but they are ok in rest of the world. Has anybody had any experience blocking a site from appearing in just google.de, bing.de and yahoo.de for example?
Technical SEO | | Salience_Search_Marketing0 -
50,000 pages or a page with parameters
I have a site with about 12k pages on a topic... each of these pages could use another several pages to go into deeper detail about the topic. So, I am wondering, for SEO purposes would it be better to have something like 50,000 new pages for each sub topic or have one page that I would pass parameters to and the page would be built on the fly in code behind. The drawback to the one page with parameters is that the URL would be static but the effort to implement would be minimal. I am also not sure how google would index a single page with parameters. The drawback to the 50k pages model is the dev effort and possibly committed some faux pas by unleashing so many links to my internal pages. I might also have to mix aspx with html because my project can't be that large. Anyone here ever have this sort of choice to make? Is there a third way I am not considering?
Technical SEO | | Banknotes0 -
I need help compiling solid documentation and data (if possible) that having tons of orphaned pages is bad for SEO - Can you help?
I spent an hour this afternoon trying to convince my CEO that having thousands of orphaned pages is bad for SEO. His argument was "If they aren't indexed, then I don't see how it can be a problem." Despite my best efforts to convince him that thousands of them ARE indexed, he simply said "Unless you can prove it's bad and prove what benefit the site would get out of cleaning them up, I don't see it as a priority." So, I am turning to all you brilliant folks here in Q & A and asking for help...and some words of encouragement would be nice today too 🙂 Dana
Technical SEO | | danatanseo0 -
Can Not Save the SEO Settings on Attachement/Media Page
I am trying to save SEO settings to a wordpress gallery attachment page for a picture. When I fill up all info and hit save all the writing disappear from from the form. Is it a software bug or there is a solution for it??
Technical SEO | | ExpertSolutions0 -
If two links from one page link to another, how can I get the second link's anchor text to count?
I am working on an e-commerce site and on the category pages each of the product listings link to the product page twice. The first is an image link and then the second is the product name. I want to get the anchor text of the second link to count. If I no-follow the image link will that help at all? If not is there a way to do this?
Technical SEO | | JordanJudson0 -
Can I use canonical tags to merge property map pages and availability pages to their counterpart overview pages?
I have a property website, for each property are 4-5 tabs each with their own URL, these pages include the overview page which is content rich, and auxilliary pages such as maps, availability, can I use a canonical tag to merge the tabs with very little content to their corresponding overview page which is content rich? I.e. www.mywebsite.co.uk/property-1/overview This page has tabs for map, town info, availability which all have their own url i.e. www.mywebsite.co.uk/property-1/map
Technical SEO | | assertive-media
www.mywebsite.co.uk/property-1/availability
www.mywebsite.co.uk/property-1/towninfo Because these auxilary pages do not contain much content can I place a canonical tag in them pointing back to the content rich overview page at www.mywebsite.co.uk/property-1/overview?0