How to use Crawl Report for a test directory
-
I have a client's new site set up in a folder that is not linked to from the main site. When setting up the Crawl Report, I put in the starting url for the new folder, http://oldsite.com/new/start.php. The Crawl Report came back with a crawl of the current site instead. How do folks run the crawl report to test sites before they are public? Thanks!
-
Mary,
Thanks for reaching out to us! That is a good question! Our crawler can only crawl sites that are indexable and that are public, so if you set the report to run against the starting URL for the new folder, it would try to find that folder if it is public. One thing you could try out is to set your site public right before a crawl date, then immediately close access to it, this is the only way I can think Roger would be able to do it
I am sorry that I can't offer something better, I hope that helps!
Best,
Peter
Moz Help Team.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site Crawl 4xx Errors?
Hello! When I check our website's critical crawler issues with Moz Site Crawler, I'm seeing over 1000 pages with a 4xx error. All of the pages that are showing to have a 4xx error appear to be the brand and product pages we have on our website, but with /URL at the end of each permalink. For example, we have a page on our site for a brand called Davinci. The URL is https://kannakart.com/davinci/. In the site crawler, I'm seeing the 4xx for this URL: https://kannakart.com/davinci/URL. Could this be a plugin on our site that is generating these URLs? If they're going to be an issue, I'd like to remove them. However, I'm not sure exactly where to begin. Thanks in advance for the help, -Andrew
Moz Pro | | mostcg0 -
Crawl report shows Title Element too long but they aren't
Hi, My latest crawl report says that I have a stack of pages with Title Element Too Long on them - e.g. Build My Ride - charity team building event with real purposeBuild My Ride - charity team building event with real purpose http://www.teamelevate.co.nz/events/build-my-ride1.html You can see that it shows the title element as doubled-up. When I look at the title element on the live page it is not double. GWT shows that there are no issues with long title elements. Any ideas anyone...? Chris
Moz Pro | | chris.elevate0 -
False Pro reporting of duplicate titles
I am testing Pro. About 250 pages of content at my website. Pro says ALL of my pages have duplicate titles., but when I click on details, they display as unique titles. Ie: first page of results of Pro is as follows. While the content of my website is on one major topic the title meta tags are NOT identical. Is this an issue with Pro, or is Pro looking at something other than the title meta tags? Please advise ? Fiance Visa Help What is Adjustment Of Status from K1 Visa Adjustment of Status support Taiwan US Consulate Visa Interview Adjustment of Status Order Form How to Choose between K1 Fiancee or CR1 Marriage Visa Removal of Conditions on Residence support US Embassies + Consulates that process Fiancee and Spousal Visas | | | | |
Moz Pro | | microonae
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| |0 -
Question regarding the SEOmoz Report Card
A report card is directed towards one keyword. What if one is to target two keywords on one page? For example, if I want to target SEO in Miami, Miami SEO, and SEO Miami it would be impossible to get an "A" on the report card for all three. What is the best way to target two-three keys on one page? Lets say these keywords are equally beneficial to the site, and we do not want to have to use hierarchy for one key over the other. Would it be wise to build a page for each? I feel like that would be redundant to the user. It says I need the keyword 4 times in the body so I should have 4 "SEO Miami" and 4 "Miami SEO". High importance factors. What makes them high importance? The keyword should be in the beginning of the title. How does this make sense as a high importance factor. Many titles are going to sound spammy if this is how they are written. Lets use Miami SEO again as an example: Miami SEO from [company name] bla bla bla sounds like spam. Learn about SEO in Miami from [company name] sound much better and is probably more likely to get clicked on. How important are these High Importance factors? I was always under the impression that we build the site for the user however the report card changes that perspective.
Moz Pro | | SEODinosaur0 -
Plurals and the SEOmoz On Page Report Card
I have some pages on my website that receive an "A" grade with the SEOmoz On Page Report Card if I use a plural term in a phrase (say, "landscape designers in Dallas") but an "F" grade if I use the singular term (say, "landscape designer in Dallas"). Is this a true and accurate reflection of how search engines see the page? Am I going to rank well for plural terms but not for singular, given the specific on-page content I'm using in this example? Or is this an opportunity for SEOmoz to tweak its' algorithm to better reflect the search engines?
Moz Pro | | ahirai1 -
Campaign Crawl Report
Hello, Just a quicky, is there anyway I can do a crawl report for something in a campaign so I can compare the changes? I know you can do a separate crawl test, but it wont show the differences,and the next crawl date isnt untill the 28th.
Moz Pro | | Prestige-SEO0 -
Why Is SEOMOZ No Longer crawling All Of My Site
Hi all, I joined Seomoz over a month ago and Roger has been crawling all of the pages on the site approx 20 pages. Through out the last few weeks I have been working on the errors and notices identified by Roger. However, this week Roger has only re-crawled 1 page and is not picking up all the other pages. Has any one come across this problem. can you recommend any thing to resolve it? Many thanks in advance....
Moz Pro | | Dan280