Hoe to crawl specific subfolders
-
I tried to create a campaign to crawl the subfolders of my site, but it stops at just 1 folder. Basically what I want to do is crawl everything after folder1:
I tried to create 2 campaigns:
Subfolder Campaign 1: www.domain.com/web/folder1/*
Subfolder Campaign 2: www.domain.com/web/folder1/In both cases, it did not crawl and folders after the last /. Can you help me ?
-
According to Moz Help, "The PRO Web App crawls, grades and checks rankings only for pages found on that domain or subfolder. When tracking a subfolder, we will only crawl pages that exist in that specific folder. We will not crawl links that go to other folders."
Hope this helps.
Mike
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Improving ranking for a more specific keyword - eg 1 word
Hi there Ive managed to get my site ranking in top5 of Organic search for a 2 word keyword "keywordname australia" for example or "buy keywordname" - however finding it a lot slower process improving rankings for just the actual 'keywordname' Any suggestions as to what might be the best thing to focus on to improve a more competitive term?
Moz Pro | | IsaCleanse0 -
404 : Errors in crawl report - all pages are listed with index.html on a WordPress site
Hi Mozers, I have recently submitted a website using moz, which has pulled up a second version of every page on the WordPress site as a 404 error with index.html at the end of the URL. e.g Live page URL - http://www.autostemtechnology.com/applications/civil-blasting/ Report page URL - http://www.autostemtechnology.com/applications/civil-blasting/index.html The permalink structure is set as /%postname%/ For some reason the report has listed every page with index.html at the end of the page URL. I have tried a number of redirects in the .htaccess file but doesn't seem to work. Any suggestions will be strongly appreciated. Thanks
Moz Pro | | AmanziDigital0 -
Crawl Report Warnings
How much notice should be paid to the warnings on the SEO Moz crawl reports? We manage a fairly large property site and a lot of the errors on the crawl reports relate to automated responses. As a matter of priority which of the list below will have negative affects with the search engines? Temporary RedirectToo Many On-Page LinksOverly-Dynamic URLTitle Element Too Long (> 70 Characters)Title Missing or EmptyDuplicate Page ContentDuplicate Page TitleMissing Meta Description Tag
Moz Pro | | SoundinTheory0 -
Moz crawling
Hi Everyone! I'm new to the SEOMoz and wanted to find out if there is a way to decrease the waiting time for the campaign crawl. I have made a lot of changes based on the first crawl and would like to see how these are reflected on the reports, but can't until the next crawl is performed. Any help would be greatly appreciated.
Moz Pro | | coremediadesign0 -
Only crawling one page
Hi there, A campaign was crawling fine, but at the last crawl, for some reason, SEOmoz can only crawl one page... any ideas? If I run a custom crawl I still access all of the site's pages.
Moz Pro | | harryholmes0070 -
Why did the crawl last night not show the same results i see in google?
Last night my keywords were crawled and it shows me that a key word is ranked 14. For 3 days now it has been rank 4 or 5. Is there a reason this is not accurate? I have not checked the rest of my keywords so i am not sure about those. Thanks
Moz Pro | | tom14cat140 -
Only 1 page is being crawled by SEOmoz for the last 2 crawls
I would like to ask for the possible problem plus solution on one of our campaigns. Only 1 page is being crawled by SEOmoz for the last 2 crawls. Before the last two crawls, SEOmoz crawls numerous pages and we can’t think of a possible reason for this error. For this particular campaign , there are no data --- no errors, warnings and notices. Thanks!
Moz Pro | | TheNorthernOffice790