How can you set SEOmoz to work with your dev site behind an htpasswd?
-
All sites need to be developed from the small to the grand - and this takes time. Development usually takes place on a subdomain different from our live domain. It is locked down behind an htpasswd during development so its not picked up by searching engines - that may create duplicate content issues if when the site goes live it has already scanned our site on the development server. Its also a security implementation to keep the site away from prying eyes before its ready for launch There could be security holes that have not been tweaked.
Whats the best strategy to get SEOmoz involved in this scenario. Its tools are invaluable to the SEO part of the build - but the seomoz crawler bot has a different IP address (being cloud based) - so we cannot just let a single IP address through our htpasswd.
Also is there a way to link the dev and live site in seomoz - so when it goes live to maintain all teh same logs without having to create two seperate site campaigns?
Thanks!
-
Hey Robert!
Yes, everything is good here, thanks. Just been hectic around here lately and yes, I've been feeling bad about missing my time in Q&A
-
Good answer there Sha. Glad to see your smiling face, thought you were hiding! Hope all is well down under.
-
Hi dseo241,
Given that your question specifically relates to Roger's ability to access your sites (both dev & production), I think you would be best to email the SEOmoz Help Team direct - help at seomoz.org.
I use the stand alone Crawl Test tool to run a base crawl of our dev sites, which are noindexed, but not behind an htpassword, so you would still need to ask the Help Team about accessibility. Incidentally, the SEOmoz User-agent is rogerbot.
Hope that helps,
Sha
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Export to csv does not work
HI friends. For long time I cannot download csv reports for links. I need to export new spam links sort by spam score and compare to existing but export to csv button spins for ever and nothing happens
Moz Pro | | netcomsia0 -
Confused on setting up my domain in a campaign
I have a website that is www.example.com. When I first setup my campaign in SEOMoz, I missed the instructions that stated I need to set up my domain as it appeared in the browser. Thus, I set it up as example.com. When I realized my mistake, I setup a second campaign as www.example.com and am running them side by side to see the differences. My question has to do with the best setup. I have a blog that is a subdomain as blog.example.com. It appears that the www.example.com campaign is not crawling the blog pages but example.com is. Are there any downsides in setting up this campaign as example.com? I have a number of other sites with the same issue so would like to get this resolved before I setup the additional sites. Thanks for any assistance or insights into ways I may be "shooting myself in the foot" with the wrong domain structure in my campaign settings.
Moz Pro | | rfwood0 -
Quote "There is no SEO software that can help optimize a site"
Hi all I am a developer for a small agency in the UK. I am using SEOmoz for a couple of our clients and I am also using it to learn from. So far I am finding it a great tool and this forum is also incredibly helpful. Now I also visit another forum called HighRankings.com and this months newsletter is a list of points that are defined as "SEO Mistakes". In it, point number 2 reads: "Repeat after me: There is no specific number of times a keyword phrase should be used in my content. There is no magic number of words that my pages should have written on them. And there is no best number of words or phrases that belong in a Title tag. And most of all: There is no SEO software that can optimize my website (despite the claims of their creators). Use your common sense to optimize your site! Learn how your target audience searches for products, services and information such as yours, and write about it accordingly on your website. Then write to make an emotional connection with your visitors so that they'll convert into happy customers." Now as a user of SEOmoz I wondered what others thought of this comment? Many thanks for reading. Kris
Moz Pro | | yousayjump0 -
Site Ranking Report
Hi guys, My site ranking report says that I've gone from being 1-20 for a variety of keywords in Google UK to not in the top 50. When I do a search myself I see that my site remains where it previously was (between 1-20). How reliable is the site ranking reporting on a weekly basis? Is it best to look at it monthly?
Moz Pro | | columbus0 -
Why Is SEOMOZ No Longer crawling All Of My Site
Hi all, I joined Seomoz over a month ago and Roger has been crawling all of the pages on the site approx 20 pages. Through out the last few weeks I have been working on the errors and notices identified by Roger. However, this week Roger has only re-crawled 1 page and is not picking up all the other pages. Has any one come across this problem. can you recommend any thing to resolve it? Many thanks in advance....
Moz Pro | | Dan280 -
I need a tool/tools to extract keywords from say 50 sites in one niche and then check the rank tor those sites
This is for telemarketing of seo services I want to have some insight into an industry before I call them could this be done with the adwords keyword tool api and then exported to excel. It would also be nice to have data on backlinks say from seo moz opensite explorer.. Its just that the research you do before you even call a potential client is so time consuming and you can never really check to see how they are ranking for there main keywords manually. We are trying to automate as much of this initial research as possible... Any Idea's Thanks
Moz Pro | | duncan2740 -
Can I exclude pages from my Crawl Diagnostics?
Right now my crawl diagnostic information is being skewed because it's including the onsite search from my website. Is there a way to remove certain pages like search from the errors and warnings of the crawl diagnostic? My search pages are coming up as: Long URL Title Element Too Long Missing Meta Description Blocked by meta-robots (Which is how I want it) Rel Canonical Here is what the crawl diagnostic thinks my page URL looks like: website.com/search/gutter%25252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 525252525252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 52525252525252525252525252525252525252525252525252Bcleaning/ Thank you, Jonathan
Moz Pro | | JonathanGoodman0 -
Can SEOMOZ do Local SEO ?
Most of our clients are local but we can not get local or regional (for example VA State or Washington DC) SERP results within SEOMOZ. it would be nice to create custom search profiles based on region (State or County) and track SERP within SEOMOZ. Ranking reports and even keyword difficulty analysis etc. within SEOMOZ are misleading for local web sites. SEOMOZ default site setup assumes every business doing SEO serves all over the US. I think it will be very beneficial if we have a choice to select State or County of local web sites during initial campaing set up at SEOMOZ. what do you think? do you know any work around of this problem? Thanks,
Moz Pro | | CertifiedSEO
Lewis2
