Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Crawl Test Tool - SEO Web Crawler work on mobile sites?

Moz Bar

1451

HelloAlba last edited by

Hi does the MOZ Crawl Test Tool - SEO Web Crawler work on mobile sites?
1 Reply Last reply
Reply Quote 0
HelloAlba @Abe_Schmidt last edited by

Thanks Abe! I should be good now.
1 Reply Last reply
Reply Quote 0
Abe_Schmidt last edited by

Hello! Yes, our crawl test tool does work with most mobile sites. I confirmed this earlier this morning as well for a member I'm working with at the moment. If you are having trouble with the tool, feel free to contact help@Moz.com with exact page examples.
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

What do you do when there is significant movement when checking a Ranking & Algorithm tracking tool?

I've checked various Ranking & Algorithm tracking tools (SEMrush, Mozcast etc.) and since 12th Jan 18 there has been a lot of 'turbulence' in terms predicted Google algorithm changes. This has been 'high' for the past 2 weeks. Question is, would you 'hang tight and ride the wave' or 'act fast'? Our actual rankings haven't really been affected positively or negatively - so I'm thinking I should just ride the wave to see what happens but keep a close eye on the rankings?
Moz Bar | | Jack_Jahan

1
Does Moz's keyword tool pull data from your IP address?

Does anyone know how Moz's keyword tool pulls their keyword ranks? Do they take it based off of the IP (history and cookies) that is being used? I am trying to find a way to collect keyword data that is neutral and not based off of my previous searches, etc. TIA
Moz Bar | | ReviveMedia

0
Weird 404 in Crawl Diagnostics

I'am getting a lot of 404 errors (196 to be precise ) - but their pattern is weird.
The page that the crawler is trying to find is (e.g):
http://www.oorbo.com/item/asufa-israeli-design-shop**/www.oorbo.com.
the linking page is** http://www.oorbo.com/item/asufa-israeli-design-shop meaning it adds to the end of the link the root URL - /www.oorbo.com. This happens in all 196 cases - trying to find a page http://www.oorbo.com/some-page/www.oorbo.com from a refferer page http://www.oorbo.com/some-page. Obviously this pages do not exist, and it's getting a 404. I've look into the pages themselves and digged into their code - It doesn't seem that the bad link is any where on the page. Did anyone came across this kind of issue? any one can point me to a solution ?
Moz Bar | | oorbo

0
Odd crawl test issues

Hi all, first post, be gentle... Just signed up for moz with the hope that it, and the learning will help me improve my web traffic. Have managed to get a bit of woe already with one of the sites we have added to the tool. I cannot get the crawl test to do any actual crawling. Ive tried to add the domain three times now but the initial of a few pages (the auto one when you add a domain to pro) will not work for me. Instead of getting a list of problems with the site, i have a list of 18 pages where it says 'Error Code 902: Network Errors Prevented Crawler from Contacting Server'. Being a little puzzled by this, i checked the site myself...no problems. I asked several people in different locations (and countries) to have a go, and no problems for them either. I ran the same site through Raven Tool site auditor and got some results. it crawled a few thousand pages. I ran the site through screaming frog as google bot user agent, and again no issues. I just tried the fetch as Gbot in WMT and all was fine there. I'm very puzzled then as to why moz is having issues with the site but everyone is happy with it. I know the homepage takes 7 seconds to load - caching is off at the moment while we tweak the design - but all the other pages (according to SF) take average of 0.72 seconds to load. The site is a magento one so we have a lengthy robots.txt but that is not causing problems for any of the other services. The robots txt is below. Google Image Crawler Setup User-agent: Googlebot-Image
Disallow: Crawlers Setup User-agent: * Directories Disallow: /ajax/
Disallow: /404/
Disallow: /app/
Disallow: /cgi-bin/
Disallow: /downloader/
Disallow: /errors/
Disallow: /includes/
#Disallow: /js/
#Disallow: /lib/
Disallow: /magento/
#Disallow: /media/
Disallow: /pkginfo/
Disallow: /report/
Disallow: /scripts/
Disallow: /shell/
Disallow: /skin/
Disallow: /stats/
Disallow: /var/
Disallow: /catalog/product
Disallow: /index.php/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /catalogsearch/
#Disallow: /checkout/
Disallow: /control/
Disallow: /contacts/
Disallow: /customer/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/
Disallow: /catalog/product/gallery/ Files Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /STATUS.txt Paths (no clean URLs) #Disallow: /.js$
#Disallow: /.css$
Disallow: /.php$
Disallow: /?SID= Pagnation Disallow: /?dir=
Disallow: /&dir=
Disallow: /?mode=
Disallow: /&mode=
Disallow: /?order=
Disallow: /&order=
Disallow: /?p=
Disallow: /&p= If anyone has any suggestions then please i would welcome them, be it with the tool or my robots. As a side note, im aware that we are blocking the individual product pages. Too many products on the site at the moment (250k plus) which manufacturer default descriptions so we have blocked them and are working on getting the category pages and guides listed. In time we will rewrite the most popular products and unblock them as we go Many thanks Carl
Moz Bar | | Arropa

0
Performed a Moz Crawl Test - Says I have 107 External Links on Homepage??

Hello Mozzers! Exactly at the title suggests, I performed a crawl test on one our sites and the report says we have 107 external links on the homepage and another 34 on one of our internal category pages. On both of these pages I can only find 7 external links, anyone know why the crawl test is saying this? And if Moz is finding these external links could google be doing the same and punishing our site for the high number of external links? Any response appreciated! Richard
Moz Bar | | Richard-Kitmondo

0
Tool for seeing targeted keywords?

Hello Friends, Is there a tool that provides a list including targeted keywords and there ranking for those keywords? For example if I type in a website url it shows me which keywords they targeted, on which page and their ranking position. Thanks for helping out a foreign noobie;)
Moz Bar | | grobro

0
Open Site Explorer classic view is gone??! where is it?

I used to use open site explorer classic view a lot for pitching clients - it was great for comparing a few competitors at once. Now it seems to have disappeared - any idea where i can find it, or find similar functionality?
Moz Bar | | miguelitomana

0
Blocked Production Site from Search Engines - How to get it Crawled by Moz Crawler

I have an 'under development' site hosted, (which is an exact replica of live site as working on to add new functionalities & modules) - but its password protected, excluded from robots.txt (Disallow) & also marked noindex on all pages in the index - so that Googlebot & other Search Engines can not crawl the site At present the development work is almost 95% completed., Now - feel like to crawl the site through SEOMOZ Roger Bot - to know the errors and all indexed urls by Rogerbot. What's the best way to get Moz Bot crawl the site - but simultaneously continue it blocking its access to Search Engines I have gone through - https://support.google.com/webmasters/answer/93708?hl=en, it says a) Save it in a password-protected directory. Googlebot and other spiders won't be able to access the content- But this way Moz will also not be able to crawl the site b) Use a robots.txt to control access to files and directories on your server - However it also says - It's important to note that even if you use a robots.txt file to block spiders from crawling content on your site, Google could discover it in other ways and add it to our index. c) Use a noindex meta tag to prevent content from appearing in our search results - It also says that a link to the page can still appear in their search results. Because we have to crawl your page in order to see the noindex tag, there's a small chance that Googlebot won't see and respect the noindex meta tag Password Protected thus seems the best way to continue blocking. However, continuing with it will also block Moz bot to crawl the site. Any suggestions Thanks
Moz Bar | | Modi

0