How Do I Scan My New Site & Grade My Work With The Robots Turned Off? For Pre-Inspection before I launch my Site?
-
I have a new site that has all the bots turned off so google can't index my site until I'm finished it. I've been working on this site for a couple months now optimizing and I was wondering if there was anyway I can run a preliminary scan on the site for my titles, URLs, Headers, Alt Tags and pretty much anything else that will grade my work and tell me if i did anything wrong? Can MOZ do this with the Bots turned off?
Thanks
-
You could block the site for all bots except rogerbot (
Mozilla/5.0 (compatible; rogerBot/1.0; http://www.seomoz.org/dp/rogerbot).
Alternative would be to use a local crawler like Screaming Frog (and modify the spider settings so that it ignores the robots.txt)
rgds,
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why does Moz only seem to be crawling a snap shot of the site I am working with?
I was wondering if anyone can help? I am working using Moz to help improve the SEO on a website I am working with, the website contains thousands of pages, yet for some reason Moz only seems to be crawling a small snap shot of the website. I know there are particular pages that I had added a couple of weeks ago - about 300 in total - and none of these were showing on the first crawl, so I did another on-demand crawl and some of these showed up then. Despite this, it says it crawled 700ish pages, but there are getting close to 20-30ish thousand live pages on the site. Any thoughts and guidance as to why they crawling may be stopping?
Getting Started | | dsmith8020200 -
How I can increase DA of my site?
Hi, I have my 4 month old blog, PA of site is 17 but DA is still 5. I don't know how to increase DA of site. Please suggest me how to increase DA of Site https://myeasygrader.com/ . Thanks
Getting Started | | markwillson0 -
Site Crawl - Crawls only homepage?
Hi Moz Comunity! Joined Moz just 2 weeks ago and slowly trying to get used to tools available in here! Great tools and info available on this site! My concern is that Site Crawl of Moz in my Campaign seems to have crawled only my homepage and no other sub-domains, is there any reason for this? FOr some reason it seems that Moz interacts only with my homepage? Even when I tried the Keyword Exlporer set on Keyword to see if any of my pages rank for any keywords, it seems only my homepage was ranking for a few keywords. It's possible my other sub-domains don't rank for any keywords yet but still, seems suspicious... I have added a link to Site Crawl that says it has crawled only 3 pages on my site, and all are my homepage... Thanks for any help! Jacob s!AlxV7sobbcgmhJB_fXcF4EPzbPSovA
Getting Started | | Shotlife_Studio0 -
Duplicate Content after Moz Site Audit
Hello folks, So I signed up for the trial version of the Moz tool and ran an initial site audit. One of the site audit results is confusing me.
Getting Started | | jjimen03
It reports that there are two pages with duplicate content ( Each page has a duplicate page with duplicate content in it).
When I take a look at what those pages are, here is what I see: mysite.com/Contact-Us.html
mysite.com/contact-us.html
( The difference in the above is the Contact and Us, the first letters are capitalized on one of the URLS) mysite.com/index.html
mysite.com Now I am confused because for one thing, I don't have 2 Contact Us html files uploaded on my hosting server.
Why is Moz seeing 2 Contact Us pages? How to remove one? Regarding my home page, why is it flagging the same page as two different pages? How to remove of them?0 -
MOZ Guru Needed for Freelance Work
Hello -- I'm new to Moz and wanted to know if any community members were available for freelance work to help me fine-tune the setup of the Moz account for one of my clients. This could lead to additional work for other clients of my advertising agency.
Getting Started | | KRT0 -
Dashboard doesn't show new data for the passed week
Did Roger leave us to search for eastereggs 😉 ? My dashboard still shows data from the initial crawl (2- 8 april), and hasn't updated since. Have I switched something off ? The next run should contain 9 - 15 april, and that period closed yesterday so I wonder what's going on ...
Getting Started | | StephDep0 -
How do I update the crawl issues & Notifications?
I have a list of errors, most relating to missing meta descriptions. I have added a meta description to a page, visited the site and viewed the source, and the meta description is now there. When I go to analyze issues, the report it gives back for the link contains the same missing meta description as previously. How do I get it to update and realize the issue has been fixed?
Getting Started | | ETGg0
