Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Will moz crawl pages blocked by robots.txt and nofollow links?

Moz Pro

650

WizardOfMoz last edited by

i have over 2,000 temporary redirects in my campaign report

redirects are mostly events like being redirected to a login page before showing the actual data

im thinking of adding nofollow on the link so moz wont crawl the redirection to reduce the notification

will this solve my problem?
1 Reply Last reply
Reply Quote 0
Ryan_Watson last edited by

Hey Oscar,

Yes, Moz will not crawl nofollowed links, so this should solve your problem.

Cheers,

Ryan Watson
Business Development Associate | Moz
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Why is Link Count smaller than Internal Links in Crawl Test report?

We recently ran the crawl test report and for most of our pages we are getting 1150 internal links but 40-50 as the link count. Why is there such a big disparity?
Moz Pro | | usdmseo

0
Functionality of SEOmoz crawl page reports

I am trying to find a way to ask SEOmoz staff to answer this question because I think it is a functionality question so I checked SEOmoz pro resources. I also have had no responses in the Forum too it either. So here it is again. Thanks much for your consideration! Is it possible to configure the SEOMoz Rogerbot error-finding bot (that make the crawl diagnostic reports) to obey the instructions in the individual page headers and http://client.com/robots.txt file? For example, there is a page at http://truthbook.com/quotes/index.cfm month=5&day=14&year=2007 that has – in the header -
<meta name="robots" content="noindex"> </meta name="robots" content="noindex"> This page is themed Quote of the Day page and is duplicated twice intentionally at http://truthbook.com/quotes/index.cfm?month=5&day=14&year=2004 and also at http://truthbook.com/quotes/index.cfm?month=5&day=14&year=2010 but they all have <meta name="robots" content="noindex"> in them. So Google should not see them as duplicates right. Google does not in Webmaster Tools.</meta name="robots" content="noindex"> So it should not be counted 3 times? But it seems to be? How do we gen a report of the actual pages shown in the report as dups so we can check? We do not believe Google sees it as a duplicate page but Roger appears too. Similarly, one can use http://truthbook.com/contemplative_prayer/ , here also the http://truthbook.com/robots.txt tells Google to stay clear. Yet we are showing thousands of dup. page content errors when Google Webmaster tools as shown only a few hundred configured as described. Anyone? Jim
Moz Pro | | jimmyzig

0
SEO MOZ Timezone

I have looked & looked but I can not figure out how to set MY timezone on SEOMOZ? The dates are all wrong in my reports ... **Organic traffic data ** Last Data Update: 04/16/2012 Crawl Diagnostics Last Crawl Completed: Apr. 20th, 2012 Next Crawl Starts: Apr. 25th, 2012 Keyword Rankings "Your keywords are updated weekly on Saturday. The last update was April 19th, 2012." Yet today is Friday the 20th 2012??? It seems like each segment of the information is updated on a differing date?
Moz Pro | | Mcarle

0
Moz crawling

Hi Everyone! I'm new to the SEOMoz and wanted to find out if there is a way to decrease the waiting time for the campaign crawl. I have made a lot of changes based on the first crawl and would like to see how these are reflected on the reports, but can't until the next crawl is performed. Any help would be greatly appreciated.
Moz Pro | | coremediadesign

0
Crawl slow again

Once again the weekly crawl on my site is very slow. I have around 441 pages in the crawl and this has been running for over 12 hours. This last happened two weeks ago (ran for over 48 hours). Last week's crawl was much quicker (not sure exactly how long but guessing an hour or so). Is this a known issue and is there anything that can be done to unblock it? Weekends are the best time for me to assess and respond to changes I have made to my site so having this (small) crawl take most of the weekend is really quite problematic. Thanks. Mark
Moz Pro | | MarkWill

0
Why are these pages considered duplicate page content?

A recent crawl diagnostic for a client's website had several new duplicate page content errors. The problem is, I'm not sure where the error comes from since the content in the webpage is different from one another. Here's the pages that SEOMOZ reported to have duplicate page content errors: http://www.imaginet.com.ph/wireless-internet-service-providers-term http://www.imaginet.com.ph/antivirus-term http://www.imaginet.com.ph/berkeley-internet-name-domain http://www.imaginet.com.ph/customer-premises-equipment-term The only thing similar that I see is the headline which says "Glossary Terms Used in this Site" - I hope that the one sentence is the reason for the error. Any input is appreciated as I want to find out the best solution for my client's website errors. Thanks!
Moz Pro | | TheNorthernOffice79

0
Different Moz Scores Between Page and Subdomain

The Moz scores for a page-specific website:: www.website.com are different than the subdomain: www.website.com/. www.website.com: Page MozRank: 4.58
Page MozTrust: 4.9 www.website.com/ Subdomain MozRank: 5.21
Subdomain MozTrust 5.18 Does that have any significance? Should I be routing pages to the " / " version? Thanks, Daniel
Moz Pro | | DanSpeicher

0
Multiple Anchor text Links to 1 Page

Hi, Currently we have this page on our site - http://www.highstreetvouchers.com/gift-vouchers/gift-vouchers-cards.jsp It contains both "Gift Vouchers " & "Gift Cards" content which are 2 keywords we are targeting for. Currently the top nav link anchor is "Gift Vouchers & Gift Cards" If we were to change the top nav to be "Gift Vouchers" & "Gift Cards" as different tabs, both linking to the same page, what might be the SEO consequences? thanks
Moz Pro | | NSJ78

0