I have recently switzed to SSL on ramxpert.dk, now Moz is not able to crawl the site anymore
-
Hi Moz
I have recently switzed to SSL on ramxpert.dk, now Moz is not able to crawl anymore.
When I look at the crawl report there is just one fault displayed:
"403 : Received 403 (Forbidden) error response for page."
How can i solve this issue asap?
I was not aware that there would be any issues with moz when activating a SSL certificate on the site.
The campaign with the problem is: ramxpert.dk.
Regards
Jesper Nielsen -
Hi Jesper,
Jo here from the Moz support team.
As WebBoost has noted our crawler isn't compatible with SNI, but the good news is that our team has been working hard on a new crawler which is in Beta right now and can handle SNI. If you're interested in being a part of the beta testing, the link to the signup page is here: http://goo.gl/forms/LCvL9Ix8JDHfbAvr1. Once you fill out that form, you will automatically be added into the Beta during the next round.
We will be rolling the new crawler out to all of our users once it's out of it's Beta phase, but I don't have an exact timeframe of when that will be at this time as it's still being worked on.
You're welcome to reach out to us at help@moz.com if you get stuck :]
Cheers!
Jo -
Thanks BeadedSEO
It seems through that the issue is that ramxpert.dk uses SNI, which isn't supported by the moz crawler yet
I've decided to sign up for the beta version of the new crawler, which I found on the link below. Hopefully this will work while they work on the final version.
https://moz.com/community/q/moz-pro-our-web-crawler-and-sites-that-use-sni
-
Have you tried updating your robots.txt file to use this;
User-agent: rogerbot
Disallow:
User-agent: dotbot
Disallow:
It's best practise to Disalow: none, rather than Allow: /
I've tried crawling your site with SEO Spider and ran in to no issues.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Links on my website do not get highlighted by Moz bar in Chrome
I've used the Moz bar for many years to quickly figure out if a link is followed or no-followed. I recently used a Wordpress plugin to build lists of subjects and excerpts of pages with appropriate links. An example is at https://www.chicagotraveler.com/chicago-parks/ This is the main Parks section page. Below the map is a set of links and descriptions of each park in this section of the site. The problem is that when I use the Moz bar to look at this page, the links are not highlighted no matter which settings I click on. Followed, No-followed, External or Internal. I've looked at the code and while there is a bit of css nearby the links, they look fairly normal. Does this mean that moz thinks there is something wrong. Do you think google will also ignore these links? Should I scrap the plugin and build and maintain these lists manually?
Moz Bar | | EdKim0 -
Why do my Moz duplicate content results show me pages with no noticeably similar content?
Sometimes the "Pages with Duplicate Content" results under Content Issues show pages that, from what I'm able to see or otherwise test, have no duplicate content, save for the same navigation that exists on all of my pages. For example, a recent issue said that the following pages had duplicate content:
Moz Bar | | rickmic
https://freezerworks.com/index.php/html/slider-overlay
https://freezerworks.com/index.php/ufaqs/what-do-i-get-with-my-purchase-of-freezerworks
https://freezerworks.com/index.php/videos/fda-and-freezerworks-2
https://freezerworks.com/index.php/lims-testing-module Even a side-by-side of the page source in a text comparison tool shows nothing but navigation and scripts used in every page. Am I not seeing something?2 -
Will Removing or Disavowing Toxic Links Improve MOZ Domain Authority?
The vast majority of the 140 domains that link to our website are very low quality directories or and other toxic links. Only about 20-30 domains are not toxic (according to Link Research Tools confirmed by out manual inspection of these links). Would removing some of these links improve of MOZ Domain Rank? What if we cannot remove them, can NOZ detect a disavow file? In general would improving the ratio between good quality and poor quality links improve domain authority? Thanks,
Moz Bar | | Kingalan1
Alan2 -
Inaccurate results in MOZ Rank Tracker
Hello, We recently ran a test on Rank Tracker and got a result indicating that the page in question was in position #17. We did a manual check on a Google USA SERP for the query, and found that we weren't in results where the tool had indicated. Does anyone have any ideas on what might be causing this discrepancy? Thanks!
Moz Bar | | Critical_Mass0 -
Ask moz staff
Was trying to crawl a page and the moz on page grader tells that URL is not accessible. The URL is accessible thorugh my mobile and desktop. Any idea what is the issue ? The URL is https://www.practo.com/singapore/dentist IGrzzVe
Moz Bar | | ozil0 -
I want to uninstall the Moz SEO toolbar. How do I do this?
I installed the Moz toolbar and I don't understand it and it covers up important parts of websites and makes them inaccessible. I want to get it off my computer. I installed it in chrome. How do I get it off?
Moz Bar | | Bonnie761 -
Moz Crawler URL paramaters & duplicate content
Hi all, this is my first post on Moz Q&A 🙂 Questions: Does the Moz Crawler take into account rel="canonical" for search results pages with sorting / filtering URL parameters? How much time does it take for an issue to disappear from the issues list after it's been corrected? Does it come op in the next weekly report? I'm asking because the crawler is reporting 50k+ pages crawled, when in reality, this number should be closer to 1000. All pages with query parameters have the correct canonical tag pointing to the root URL, so I'm wondering whether I need to noindex the other pages for the crawler to report correct data?: Original (canonical URL): DOMAIN.COM/charters/search/mx/BS?search_location=cabo-san-lucas Filter active URL: DOMAIN.COM/charters/search/mx/BS?search_location=cabo-san-lucas&booking_date=&booking_days=1&booking_persons=1&priceFilter%5B%5D=0%2C500&includedPriceFilter%5B%5D=drinks-soft Also, if noindex is the only solution, will it impact the ranking of the pages involved? Note: Google and Bing are semi-successful in reporting index page count, each reporting around 2.5k result pages when using the site:DOMAIN.com query. The rel canonical tag was missing for a short period of time about 4 weeks ago, but since fixing the issue these pages still haven't been deindexed. Appreciate any suggestions regarding Moz Crawler & Google / Bing index count!
Moz Bar | | Vukan_Simic0 -
Can you give some examples of MOZ Local Partners who have an API and update quickly?
In answering questions about the speed of MOZ Local, I've seen this a few times: "The time it takes our partners to update the listing really depends on whether or not they have an API. If they do have an API, the listing will be updated pretty much immediately." Can someone provide a list of which Partners have an API, and which do not?
Moz Bar | | irapasternack0