Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Crawl Diagnostics: How many pages (deep) will it crawl for dup content

Moz Bar

1342

tdawson09 last edited by

Does anyone know how deep the crawl diagnostics will crawl when searching for dup content? Will it crawl the entire site, or will it only crawl "x" amount of pages?

Thanks!
1 Reply Last reply
Reply Quote 0
DavidLee last edited by

Hello!

The standard and medium plans will each have a set limit to crawl up to 50,000 pages. The higher plans have adjustable limits, 10,000, 20,000, etc.

Hope this helps
1 Reply Last reply
Reply Quote 2
OlegKorneitchouk last edited by
The number of pages crawled depends on which plan you have - https://moz.com/products/pricing
- Standard = 250k
- Medium = 500k
- Large/Premium = 1.25m
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

If we put the disavow links in google, does MOZ crawl the same links?

I have put bad or spam links in disavow file, but still showing in MOZ backlinks. So, I want to know that Why is MOZ not removing the spam links from their system?
Moz Bar | | insidewebanalytics

0
How to find the pages in the site which is ranking for zero keyword

I have around three thousand pages in my website,how to find the list of pages which is ranking for zero keywords
Moz Bar | | srinivasan.n

1
Crawl report shows that it gets 4xx errors for pages that work fine. Why?

On the crawl report it has all these "Critical Crawler Issues". They all say "4xx Error", yet when i click on the link from the crawler report, it goes to a perfectly functioning page, not a 404 page or anything. If i click in it actually says it's a 403 error. It's all for pages generated by the IDX solution for our real estate website. Is Moz broken or am i missing something? Here are a couple examples: <dl class="crawl-page-details-list"> <dd class="crawl-page-details-list-emphasis">https://teamvivi.com/homes-for-sale-map-search/</dd> <dd class="crawl-page-details-list-emphasis"> <dl class="crawl-page-details-list"> <dd class="crawl-page-details-list-emphasis">https://teamvivi.com/email-alerts/</dd> </dl> </dd> </dl>
Moz Bar | | TeamViviRealEstate

0
Weird 404 in Crawl Diagnostics

I'am getting a lot of 404 errors (196 to be precise ) - but their pattern is weird.
The page that the crawler is trying to find is (e.g):
http://www.oorbo.com/item/asufa-israeli-design-shop**/www.oorbo.com.
the linking page is** http://www.oorbo.com/item/asufa-israeli-design-shop meaning it adds to the end of the link the root URL - /www.oorbo.com. This happens in all 196 cases - trying to find a page http://www.oorbo.com/some-page/www.oorbo.com from a refferer page http://www.oorbo.com/some-page. Obviously this pages do not exist, and it's getting a 404. I've look into the pages themselves and digged into their code - It doesn't seem that the bad link is any where on the page. Did anyone came across this kind of issue? any one can point me to a solution ?
Moz Bar | | oorbo

0
What is Considered Duplicate Content by Crawlers?

I am asking this because I have a couple of site audit tools that I use to crawl a site I work on every week and they are showing duplicate content issues (which I know there is a lot on this site) but some of what is flagged as duplicate content makes no sense. For example, the following URL's were grouped together as duplicate content: | https://www.firefold.com/contact-us | https://www.firefold.com/gabe | https://www.firefold.com/sale | | | How are these pages duplicate content? I am confused on what site audit tools are considering duplicate content. Just FYI, this is data from Moz crawl diagnostics but SEMrush site auditor is giving me the same type of data. Any help would be greatly appreciated. Ryan
Moz Bar | | RyanRhodes

0
Problem Downloading Crawl Error Report PDF's

I am trying to download the PDF reports for the various 'crawl errors' - now some of them are quite large but would that justify why I am unable to download - the error is a straightforward one, see attached. Any ideas? Andy aDlViIN
Moz Bar | | TomKing

0
"Sorry! We weren't able to find that page when we crawled your site." Please help!

Can someone please explain whey I am getting this error for this link "http://lensoutloud.com/san-antonio-real-estate-photography/" when I attempt to perform an on page SEO grading? The link is indexed and ranking very well but for some reason Moz says it can't find the page when it crawled my site. This has also happened when I attempt to grade other pages on my site. Thanks in advance!
Moz Bar | | AndreGant

0
Is it no longer possible to export the On-Page grader reports?

Hi Everyone, I was just trying to export the On Page Grader report for my clients website, and found that there is no longer that option. Is this a permanent change with the new branding? Would appreciate any feedback. Thanks very much.
Moz Bar | | Peter.Huxley59

0