I have recently switzed to SSL on ramxpert.dk, now Moz is not able to crawl the site anymore
-
Hi Moz
I have recently switzed to SSL on ramxpert.dk, now Moz is not able to crawl anymore.
When I look at the crawl report there is just one fault displayed:
"403 : Received 403 (Forbidden) error response for page."
How can i solve this issue asap?
I was not aware that there would be any issues with moz when activating a SSL certificate on the site.
The campaign with the problem is: ramxpert.dk.
Regards
Jesper Nielsen -
Hi Jesper,
Jo here from the Moz support team.
As WebBoost has noted our crawler isn't compatible with SNI, but the good news is that our team has been working hard on a new crawler which is in Beta right now and can handle SNI. If you're interested in being a part of the beta testing, the link to the signup page is here: http://goo.gl/forms/LCvL9Ix8JDHfbAvr1. Once you fill out that form, you will automatically be added into the Beta during the next round.
We will be rolling the new crawler out to all of our users once it's out of it's Beta phase, but I don't have an exact timeframe of when that will be at this time as it's still being worked on.
You're welcome to reach out to us at help@moz.com if you get stuck :]
Cheers!
Jo -
Thanks BeadedSEO
It seems through that the issue is that ramxpert.dk uses SNI, which isn't supported by the moz crawler yet
I've decided to sign up for the beta version of the new crawler, which I found on the link below. Hopefully this will work while they work on the final version.
https://moz.com/community/q/moz-pro-our-web-crawler-and-sites-that-use-sni
-
Have you tried updating your robots.txt file to use this;
User-agent: rogerbot
Disallow:
User-agent: dotbot
Disallow:
It's best practise to Disalow: none, rather than Allow: /
I've tried crawling your site with SEO Spider and ran in to no issues.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Links and images in custom css are not recognised by Moz Toolbar or Yoast plugin
Hi, I am building a website in Wordpress and when checking the pages through Moz toolbar or Yoast plugin, I notice that these don't recognise the internal links in custom CSS and the images neither. Here some code to clarify how it is built: [vc_row][vc_column width="1/2"][ultimate_heading main_heading="Veldrijden" alignment="left" main_heading_margin="margin-bottom:20px;"][/ultimate_heading][vc_single_image image="6497" img_size="full" onclick="custom_link" link="http://k488239.hosting.kinamo.be/activiteiten/veldrijden/"][vc_btn title="Meer info" color="mulled-wine" align="left" link="url:http%3A%2F%2Fk488239.hosting.kinamo.be%2Factiviteiten%2Fveldrijden%2F|||"][/vc_column] The links are embedded in images and in 'More info' buttons. Also the internal links in the main website navigation are not recognised, but the normal a href links are recognised. For the images it is a little different: the alt text from the images gets recognised by Moz, but Yoast says there are no images on the page. The only thing I can think of that there is no 'a href' in the links and for the images that it has something to do with the page builder. Can this effect my SEO results? Thanks.
Moz Bar | | Mat_C0 -
How can I find duplicate pages from a Moz Crawl?
We have many duplicate pages that show up on the Moz Crawl, and we're trying to fix these but it's very difficult because I can't see a way to isolate the code where the duplicate is found. For instance, http://experiencemission.org/immersion/ is one of our main pages, and the crawl shows one duplicate of http://experiencemission.org/immersion. It appears that one of our staff manually edited the source code in one of our pages but forgot the trailing slash. This would be an easy fix but the problem is that this page is linked to internally on our website 2423 times, so it's next to impossible to find the code that is incorrect. We have many other pages with this same basic problem. We know we have duplicates, but it's next to impossible to isolate them. So my question is this: When viewing the Moz Crawl data is there any way to see where a specific duplicate page link is located on our website? Thanks for any and all help!
Moz Bar | | expmission0 -
Crawl Test Takes Long Time
Hi Moz, I have submitted our website for a crawl test. Usually it would only take a few hours to do the crawl. However this time, it takes quite long time and the result still shows in progress 😞 This is a small website which only contains less than 10 pages. Just wondering if this is our website setting issue or it is a technical issue at your end? Many thanks in advance. sFjAERG.png
Moz Bar | | russellbrown0 -
How old is this Moz quiz?
Does anyone know? I want to use it as a study tool if the info is still current. SEO Expert Quiz
Moz Bar | | SSFCU
http://moz.com/seo-expert-quiz Thanks,
Sarah0 -
MOZ crawl test is not reporting on all the pages on my site.
I've run the crawl test one of the sites I've taken over SEO for, however its only picking all the pages. For instance it indexes all the pages under xxxxx/us but none under xxxxx/au or xxxxx/uk The pages are being indexed as they're ranking in Google. Thanks.
Moz Bar | | ahyde0 -
Crawl Test cannot be seen on my PC. Using Windows 8.
I received and downloaded my Crawl Test. When I try to open it, my pc says "This app can't run on your PC. To find a version for your PC, check with the software publisher". I'm running Windows 8. Can I view my Crawl Test with my PC? Is there a work-around for this issue? Update I can apparently open my Crawl Test and view it as an Excel Spreadsheet. But when I download it and choose Save As, it saves it as a MS-DOS Application. This is my very first Crawl Test and I am not sure if I am doing everything right.
Moz Bar | | jameskoby010 -
Site Wide Content Analysis
Are there tools that can effectively analyze your content site wide and provide recommendations?
Moz Bar | | SpadaMan0 -
Can the Moz tool identify variations of a Chinese language branded keyword?
We've recently started a trial of the Moz Pro service and are tracking a selection of keywords. Our primary band / product name is 功夫英语(Kungfu English), so we've set a rule that any keywords containing those four Chinese characters (功夫英语) should be marked as a branded keyword. However, in the "Non-Paid Keywords Sending Search Visits" section of our traffic report, we see a few variations on our brand name that are not being marked as branded keywords. (See attached Images). Based on our rules, shouldn't these variations also be marked as branded keywords without our needing to manually add them as such? Or have I misunderstood the intent of this rule? For the English text, the brand rule about words containing "kungfuenglish" seems to have resulted in all of "www.kungfuenglish.com", "http://www.kungfuenglish.com", and kungfuenglish being labelled as branded keywords. However, I'm not seeing the same sort of result with variations on the Chinese keyword, 功夫英语. KedTi1D.png z5bcUIt.png
Moz Bar | | PaulCoffey0