Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Scheduling crawls between certain time periods

Moz Pro

2237

RugsUSA last edited by

Hi, today SEOMoz crawled our site and it interfered with an email campaign that we sent out and pretty much brought our site to a crawl (seoMoz even reported numerous 4XX errors).

Is there a way to tell the crawler to only allow indexing between certain time periods?
1 Reply Last reply
Reply Quote 0
elephantseo last edited by

There is no way to do it from the web application interface and in my talks with SEOmoz in the past there is no way to set to a specific time or day. I'd recommend either calling or sending a request to customerservice@seomoz.org. Phone number is on http://www.seomoz.org/about/contact
1 Reply Last reply
Reply Quote 2

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Help talking to an aggressive web team (1st time in the forums)

Hello All! I am a long-time Moz user, but not super technical myself. I'm used to asking a web team to fix something, based on the reports, and they do it. I'm trying to communicate with a really aggressive web team about a site that's showing about 16,000 errors every week on my Moz reports. What I'm trying to figure out is if they are right or not, so I can either ignore them or push harder. Can someone help me with how to explain these to them or let me know if they are right? These are the two biggies I'm struggling with: b. Duplicate page content
Example: https://www.whiteheadindustrial.com/c-104-abrasives.aspx?pagenum=1 and https://www.whiteheadindustrial.com/c-104-abrasives.aspx Their explanation: I wonder if MOZ is not reading the infinite scrolling pages the way that Google does. We have followed Google’s directions for how to let Google know that each page is a successor of the previous one, so no duplicate content. But probably MOZ tool does not view the code on the page the same way. So I think this is not relevant for Google indexint. Second:
Example 2: https://www.whiteheadindustrial.com/p-19708-4-point-mandrel-and-hex-nut-116-hole-18-shank-nocra-13440-m.aspx
https://www.whiteheadindustrial.com/p-30356-working-peanut-knife-no-046-6220-ss.aspx
In Example 2, the only duplicate content I can find is the “Stocking data Current as of… Please note:..” and the Customer Reviews area. c. Duplicate page title
I wonder if MOZ is not reading the infinite scrolling pages the way that Google does; because the duplicate page titles are related to category page I would be super grateful for any guidance!
Moz Pro | | Renia

1
Still Cant Crawl My Site

I've removed all blocks but two from our htaccess. They are for amazonaws.com to block amazon from crawling us. I did a fetch as google in our WM tools on our robots txt with success. SEOMoz crawler here hit's our site and gets a 403. I've looks in our blocked request logs and amazon is the only one in there. What is going on here?
Moz Pro | | martJ

0
Summarize your question.Is it possible to request another unscheduled crawl?

We have just sorted a couple of issues on the website which threw the crawl into spasm and gave us hundreds of hugely long URLs. We are pretty sure that we have corrected this and do not want to wait another week to check what SEOMOZ comes up with. Is there anyway that we can request a special crawl of the website so that we can hopefully just be left any legitimate remaining issues?
Moz Pro | | dmckenzie456

0
Why have I stopped receiving emails about crawl reports and rankings reports?

I used to receive emails weekly telling me a new crawl had completed and the reports were ready and also another email saying the new rankings and on page reports were ready to view - I am not getting these anymore. Does this mean these are still happening but I need to find them within the package or has something changed? Thanks
Moz Pro | | Fitsensesports

0
Crawl Diagnostics Shows thousands of 302's from a single url. I'm confused

Hi guys I just ran my first campaign and the crawl diagnostics are showing some results I'm unfamiliar with.. In the warnings section it shows 2,838 redirects.. this is where I want to focus. When I click here it shows 5 redirects per page. When I go to click on page 2, or next page, or any other page than page 1 for that matter... this is where things get confusing. Nothing shows. Downloading the csv reveals that 2,834 of these are all showing: URL: http://www.mydomain.com/401/login.php url: http://www.mydomain.com/401/login.php referrer: http://www.mydomain.com/401/login.php location_header: http://www.mydomain.com/401/login.php I guess I'm just looking for an explanation as to why it's showing so many to the same page and what possible actions can be taken on my part to correct it (if needed). Thanks in advance
Moz Pro | | sethwb

0
Third crawl of my sites back to 250 pages

Hi all, I've been waiting some days for the third crawl of my sites, but SEOMOZ only crawled 277 pages. The next phrase appeared on my crawl report: Pages Crawled: 277 | Limit: 250 My last 2 crawls were of about 10K limit. Any idea? Kind regards, Simon.
Moz Pro | | Aureka

0
Crawl Diagnostics shows two title and meta tag errors but they are false positives.

I got one hit each on "Missing Meta Description Tag" and "Title Missing or Empty" but in the source of my page they are clearly there: <title>Protein Powder | Compare and Get the Best Prices</title> <meta name="keywords" content="protein powder, whey protein, protein supplement, whey protein isolate, hydrolyzed whey" /> I understand there are conventions which may or may not be followed by Drupal (I read an earlier question where ordering and W3C conventions were suggested) but i'm not sure how to fix them given Drupal will just overwrite any hand editing the next time something is built and importantly, I can't get the crawl to work on cue - it works on the automatic once a week crawl in the main campaign summary but every time I've specifically used the Crawl Test tool it gives me a "There was an error submitting your request to the crawler. Please try again later" so I can't really test any changes. Given Google seems to be recognising the title tag - ie showing it in the results - Do I put this down as seomoz just not working? Kind Regards, Brian
Moz Pro | | btrr69

0
Crawl Diagnostics bringing 20k+ errors as duplicate content due to session ids

Signed up to the trial version of Seomoz today just to check it out as I have decided I'm going to do my own SEO rather than outsource it (been let down a few times!). So far I like the look of things and have a feeling I am going to learn a lot and get results. However I have just stumbled on something. After Seomoz dones it's crawl diagnostics run on the site (www.deviltronics.com) it is showing 20,000+ plus errors. From what I can see almost 99% of this is being picked up as erros for duplicate content due to session id's, so i am not sure what to do! I have done a "site:www.deviltronics.com" on google and this certainly doesn't pick up the session id's/duplicate content. So could this just be an issue with the Seomoz bot. If so how can I get Seomoz to ignore these on the crawl? Can I get my developer to add some code somewhere. Help will be much appreciated. Asif
Moz Pro | | blagger

0