Crawl test from tools
-
Hi,
I notice that the crawl test which is from the Research Tools doesn't really get a new crawl even though there is 2 crawl per day. It will only provide the data which was acquire from the crawl diagnostics in my pro account. There is no point for me to get the data which I get from my crawl diagnostic isn't it? Even seomoz provided with more than 2 crawl per day also useless in this case.
This whole thing doesn't make sense as the crawl diagnostics will only perform a full crawl test once every week. but even the crawl test also not helping any thing out for me.
-
Everyone, thanks for helping... Greatly appreciated.
-
Hi there Hanzo,
Thanks for reaching out! it looks like you are having some trouble with the crawl test. Keri is correct that our crawl test uses "Cached" information to construct your crawl, which can take up to 2 days to clear. While I am not sure why the sites are still reporting 404s, there's a few things that could explain why you might be getting a stale crawl:
- Your server simply did not update the changes you've made on the backend.
- There might an issue with our crawler, which in my last status there isn't a outage that I know of.
I looked into your crawl data and the 404s seems to be confirmed by the report, if you download your latest report and do a sort you should see that
Hope I was able to provide some insight, please let me know if you have questions!
Best,
-
I believe the crawl test has a cache of a couple of days. I'm asking someone from the help desk to come in and confirm, but it may be Monday before they have a chance to do so.
-
I know there is third party app which I can use to crawl,
but then isn't the crawl test is there for us to do testing instead of using other app rather than seomoz's tool?
-
You may want to give screaming frog or xenu's link sleuth. They should be able to crawl you site and give you the info you need.
-
From what I do with the crawl test, it doesn't show me the latest crawl. I need to check whether all the error 404 or related error was being solved. It only show me the old data which was crawl before but not the latest one.
-
Hanzo, what is it that you need to do that the crawl test isn't helping you with?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My moz only one page was crawled
I recently moved my shopping cart from one provider to another and today moz only crawled one page, could this be because maybe google has not indexed it yet or should i be concerned? I pointed the DNS at the new cart monday night if that helps. I would have expected it to be indexed by now
Moz Pro | | SmartVapes0 -
Why have I stopped receiving emails about crawl reports and rankings reports?
I used to receive emails weekly telling me a new crawl had completed and the reports were ready and also another email saying the new rankings and on page reports were ready to view - I am not getting these anymore. Does this mean these are still happening but I need to find them within the package or has something changed? Thanks
Moz Pro | | Fitsensesports0 -
Can I rely on Keyword Difficulty tool?
I just ran into a problem that I hadn't expected. Testing the Keyword Difficulty I saw the results contained a result for a page that has Domain Authority=1 and Page Authority=1. As a result, Keyword Difficulty was reduced (compared to last month), which may actually be reversed if the site is crawled. Sadly, I didn't run the report on the figures as it was a small project. Questions: Can I rely on results shown by Keyword Difficulty? Are results where Domain Authority =1 are used to calculate Keyword Difficulty? If so, why is that? Is there any difference between a page that has received no links and a page that OSE/Mozscape has no link data for? The problem Using the Keyword Difficulty tool, I found swings of up to 14% in Keyword Difficulty (between Oct -Nov). Dr Pete may suggest that this is because of changes in Google's index ( http://www.seomoz.org/blog/a-week-in-the-life-of-3-keywords ). However, It would be helpful to have a figure for Keyword Difficulty that isn't affected by the gaps in the Mozscape data. The (bad) solution You can mirror something close to Keyword Difficulty using: =(Sum of Page Authorities + Sum of Domain Authorities )/20 Right now, I have resorted to manually calculating keyword difficulty. I use the SEOMoz Page Authority & Domain Authority figures and a quick splash of Excel SUMIF and COUNTIF. I find the results don't look as 'easy' when I can ignore results where the data is unknown (PageAuthority=1 & DomainAuthority = 1). Background Info One result I still have a report on is for the phrase [fixing your business puzzle] using US Results on Google. For the specific result, I found the additional information about the site: DNS lookup shows the domain was registered in 2010
Moz Pro | | Darroch
Archive.org shows no records
OSE shows no data for the site
Site uses https
Google showing No links
Robots.txt file seems fine
No Sitemap.xml0 -
Set crawl frequency
Current crawl frequency is weekly, is it possible for me to set this frequency our-self?
Moz Pro | | bhanu22170 -
Problem with the keyword difficulty tool
sorry if this has been asked/been written about previously, I had a search but couldn't find anything. I am using the keyword difficulty tool to narrow down on some keywords and whilst the % score for each keyword is being displayed, the Google Adwords data is blank. It just shows the animation as if it was pulling the data but after some ten mins or so not data is pulled, just the animation still running. Is there a problem with the tool or does the problem lay at my end? Thanks, Carl
Moz Pro | | Grumpy_Carl0 -
Sub-domain not crawled
One of our sites was recently re-designed. The home page is a landing page (www.labadieauto.com) and I moved the blog to this domain (labadieauto.com/blog/) and put a link is the bottom left of the home page. Since the change the SEOMOZ campaign overview is showing only 1 page crawled. This is not setup as a sub-domain so why isn't it showing in the crawl? Help!
Moz Pro | | LabadieAuto0 -
Clearing the Tool Reports You've Run
Is there anyway of clearing the 'Tool Reports You've Run' list on the Pro Dashboard, or setting certain reports not to be saved here?
Moz Pro | | BigMiniMan0 -
SEOmoz LDA tool experience?
http://www.seomoz.org/labs/lda Does anyone presently use this tool as a regular part of their SEO analysis? Does anyone have any updates regarding this tool? It is still in the lab despite no updates for over 6 months. Is it being retired? Or promoted to production? If you do use the tool, what % of relevance is your goal when optimizing pages?
Moz Pro | | RyanKent0