Can I crawl a password protected domain with SEOmoz?
-
Hi everyone,
Just wondered if anybody has been able to use the SEOmoz site crawler for password protected domains? On Screaming Frog you are prompted for the username and password when you set the crawler running, however SEOmoz doesn't. It seems you can only crawl sites that are live and publicly available - can anyone confirm if this is the case?
Cheers,
M
-
Hi,
I don't think it is possible to crawl any password protectec pages. Password protected pages are usually blocked and bots are unable access those pages.
And yes it is true that only sites that are publicly and live can be crawled.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to redirect old domain to new domain.
We just recently signed up to Moz with hopes of fixing our Moz Ranking. We have an old domain - http://at-net.net and a new domain - https://www.expertip.net We have set up 301 (Permanent) redirects from all pages on the old site to the new, but aren't getting the ranking or aren't getting recognized from external links to the old sites. I've read the moz article on 'Link Juice' and followed those practices, but it doesn't seem to help. Does anyone have advice on doing this? Thanks in advance,
Technical SEO | | greg.lanier
Greg0 -
Could the EMD update affect my domain?
My domain is: http://www.freescrabbledictionary.com/ "Scrabble Dictionary" is a huge keyword in my niche where I used to rank top 4. Do you see this domain as possibly being hit by the EMD? My Google Analytics does not show that I was initially hit back in Sept 2012 when it first same out.
Technical SEO | | cbielich0 -
Cross domain canonical issue
Hello fella SEOs! I have a very intriguing question concerning different TLDs across the same domain. For eg: www.mainwebsite.com, www.mainwebsite.eu, www.mainwebsite.au, www.mainwebsite.co.uk etc... Now, assuming that all these websites are similar in terms of content, will our lovely friend Google consider all these TLDs as only one and unique domain or will this cause a duplicate content problem? If yes, then how should I fix it? Thnx for your precious help guys!
Technical SEO | | SEObandits1 -
How can I use MOZ to investigate a my recent drop in domain authority?
Between MOZ's last scan of my site and the one before my domain authority dropped from 35 to 29. I'm not sure where to begin investigating this and how I can leverage MOZ in this case. Any guidance would be greatly appreciated.
Technical SEO | | bearpaw0 -
We just recently moved site domains, and I tried to set up a new campaign for the new root domain, but it threw an error?
It threw an error saying we cannot access the SERPs of this site? Any reason why? It is an https:// site instead of the http://, but even our older domain had an https://
Technical SEO | | josh1230 -
Should we 301 redirect our old domain to the new domain
We have a product that when started was under the domain appnowgo.com. We've since changed the name and the domain is now knackhq.com. The latter domain doesn't rank nearly as well as the former for many of the keywords we are targeting. For example... "online database builder" and "web app builder" are two of those keywords. Obviously having app in the domain is not a bad thing but it is our old name. The question is, should we 301 the appnowgo.com domain to knackhq.com? Or should we use that better rank and just link users to knackhq.com from the appnowgo.com site until we can increase our ranking for knackhq.com? We don't plan to update the content on appnowgo.com anymore and we obviously don't want to drop off rank if at all possible. Thanks! Eric
Technical SEO | | sitestrux0 -
Moving domain.com to subdomain.domain.com
Hi, I like to do something, but first like to take some opinions from seomoz. My question is: 1. I have a domain: brandtrends.com and i like to move this from brandtrends.com to trends.brand.com because: brandtrends.com is on the position #15 on the second page of SERP for my "brand" keyword. I like to move it under trends.brand.com but all inbound-links are @brandtrends.com What do you thing, if i move permanently 301 from brandtrends.com to trends.brand.com does it rank under brand.com on the 1st page of SERP..? like to rank trends.brand.com under brand.com on results page...!!! I have the backlinks of brandtrends.com on my hands too, should i leave the inbound links @brandtrends.com and the new ones i build with trends.brand.com or should i change the inbound links from brandtrends.com to trends.brand.com Hope you got it! THanks
Technical SEO | | leadsprofi0 -
Top Level Domains
Howdy Everyone, I have a website that will span multiple countries. The content served will be different for each country. As such, I've acquired the top level domains for different countries. I want to map the cop level domains (e.g. domain.co.uk) to uk.domain.com for development purposes (LinkedIn does this). I'm curious to know whether this is adviseable and if mapping a country-specific TLD to a subdomain will maintain local SEO value. Thanks!
Technical SEO | | RADMKT-SEO0
