How can you set SEOmoz to work with your dev site behind an htpasswd?
-
All sites need to be developed from the small to the grand - and this takes time. Development usually takes place on a subdomain different from our live domain. It is locked down behind an htpasswd during development so its not picked up by searching engines - that may create duplicate content issues if when the site goes live it has already scanned our site on the development server. Its also a security implementation to keep the site away from prying eyes before its ready for launch There could be security holes that have not been tweaked.
Whats the best strategy to get SEOmoz involved in this scenario. Its tools are invaluable to the SEO part of the build - but the seomoz crawler bot has a different IP address (being cloud based) - so we cannot just let a single IP address through our htpasswd.
Also is there a way to link the dev and live site in seomoz - so when it goes live to maintain all teh same logs without having to create two seperate site campaigns?
Thanks!
-
Hey Robert!
Yes, everything is good here, thanks. Just been hectic around here lately and yes, I've been feeling bad about missing my time in Q&A
-
Good answer there Sha. Glad to see your smiling face, thought you were hiding! Hope all is well down under.
-
Hi dseo241,
Given that your question specifically relates to Roger's ability to access your sites (both dev & production), I think you would be best to email the SEOmoz Help Team direct - help at seomoz.org.
I use the stand alone Crawl Test tool to run a base crawl of our dev sites, which are noindexed, but not behind an htpassword, so you would still need to ask the Help Team about accessibility. Incidentally, the SEOmoz User-agent is rogerbot.
Hope that helps,
Sha
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Can I exclude subdomain from a Campaign?
Also, I would like to track a subdomain without the root domain, is that possible?
Moz Pro | | AndreVa0 -
How to set the crawler or reports to ignore
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
Moz Pro | | Str82u0 -
Using Seomoz for Site Evaluation am I up to par ?
Just wanted to see how people using the seomoz bar would rate a four month old site with Domain-Homepage Authority of 27 Mozrank of 5.08 and Moztrust of 5.65 . I've read up on all the factors but just wanted to know if Im up to par on building a great site thats search engine friendly. Inner pagers are on a PA of 20 and around the same mozrank and moztrust levels of +- 5.
Moz Pro | | NikolasNikolaou0 -
Can I prevent some pages from being crawled from SEOMoz spider and still not affect Google Spider?
Well, basically that's the question 😄 Can I prevent some pages from being crawled from SEOMoz spider and still not affect Google Spider? This is, I have more than 10.000 pages on the website, and I am not interested in having reports for many of them, but I still wanna get SEO visits on them, so I want Google to crawl it easily... Thanks!
Moz Pro | | MattDG0 -
SEOMoz Q&A having some issues?
Apologies if this has been asked. I was browsing Q&A and got a Ruby error page, reloaded and everything was fine. However a bunch of the topics are kind of messed up, and it looks like a chunk of them are missing now. For example my 'questions I've answered' section of My Q&A only has the newest question and the oldest question, and when I sort the SEOMoz Tools category by newest I get one recent post and then the next one is from November of last year. You guys having database problems?
Moz Pro | | icecarats0 -
Sites Blocking Open Site Explorer? Penguin related.
Last week I was looking at a competitors site who has a link scheme going on and I could actually check the links for each anchor text. This week they don't work at all, do you think they're blocking the rogerbot on their domains? Or is there a problem with open site explorer? http://www.opensiteexplorer.org/anchors?site=www.decks.ca If you're interested in the background, all the links are to instant-home-biz . com which then redirects decks . ca - it's a tricky technique. Pretty much all of the links are from sketchy sites like: airpr23.xelr8it.biz/ airpr23.anzaland.net/ airpr23.vacation-4-free.com/airpr23.blogfreeradio.net/airpr23.blogomatik.com/http://www.morcandirect.com/mortgages/resources2.php which I thought Penguin was supposed to catch…
Moz Pro | | BeTheBoss0 -
Can i give other accounts access
I would like to be able to give limited access to members of our team so they can see SEO campaign results and print off reports without being able to edit the campaigns. Is this possible?
Moz Pro | | wouldBseoKING0 -
Site is penalized yet Google WTC and SEOMOZ says it's ok!
My first question on this community.. A high traffic site of mine has been severely hit for over two months. I took a look at GWTC and noticed a lot of suggestions on the HTML suggestions tab, where I believed that one or a combination of these errors/notices might been causing the penalty. At the moment of typing GWTC shows no HTML suggestions and minor meaningless crawl errors. Checking on SEOMOZ reported issues, I see zero errors except for a high number of duplicate page titles resulting from tag pages being accessible in the same tag title in paginations (tag.html?page=x) which can be fixed easily by using rel=canonical but I just know it can't cause such penalty. For reported warnings, site has a decent amount of title element being too long and missing meta tag description tags. I have been very active in the past with 301 inner pages of the site to make pages URLs as unique as possible and results were great, but that was long time ago before taking the hit. The penalty the site suffers from is zero rankings for newly added contents and old contents as well. It has also lost all rankings for its money terms. Site is not banned and still ranking top for its name which is a good thing. I am not sure where else I should be looking and will appreciate any good advices. Thanks!
Moz Pro | | HossamHossny0