Site Crawling with Firewall Plugin
-
Just wondering if anyone has any experience with the WordPress Simple Firewall plugin. I have a client who is concerned about security as they've had issues in that realm in the past and they've since installed this plugin: https://wordpress.org/support/view/plugin-reviews/wp-simple-firewall?filter=4
Problem is, even with a proper robots file and appropriate settings within the firewall, I still cannot crawl the site with site crawler tools.
Google seems to be accessing the site fine, but I still wonder if it is in anyway potentially hindering search spiders.
-
I would contact the developer of the plugin for those types of questions. He seems very responsive to people's input on the discussion section of his plugin. They do serve practical uses though, like preventing comment spam and such, plus your client has been specifically targeted in the past so it seems it is a value add moving forward.
-
Thanks Ryan. I guess my main concern is not so much running crawlers on the site, but making sure that search engine spiders have completely unrestricted access. In the settings for this plugin you can tell it to "try to ignore search bots". The word try, doesn't really sound all that reassuring to me. Are these type of firewalls even effective at prohibiting attacks? I'd rather just get rid of the plugin all together if the risk vs reward doesn't add up.
-
Hi Jay. You might have to disable this plugin when crawling the site as it would also create a heavier load than usual when crawling with it on (even if you are able to crawl while it's on). So a possible interim solution is disable at a low traffic time of day, crawl, then re-enable. Cheers!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site Not Being Indexed
Hey Everyone - I have a site that is being treated strangely by google (at least strange to me) The site has 24 pages in the sitemap - submitted to WMT'S over 30 days ago I've manually triggered google to crawl the homepage and all connecting links as well and submitted a couple individually. Google has been parked the indexing at 14 of the 24 pages. None of the unindexed URL's have Noindex or follow tags on them - they are clearly and easily linked to from other places on the site. The site is a brand new domain, has no manual penalty history and in my research has no reason to be considered spammy. 100% unique handwritten content I cannot figure out why google isn't indexing these pages. Has anyone encountered this before? Know any solutions? Thanks in advance.
Technical SEO | | CRO_first0 -
Why is my site not being indexed?
Hi, I have performed a site:www.menshealthanswers.co.uk search on Google and none of the pages are being indexed. I do not have a "noindex" value on my robot tag This is what is in place: Any ideas? Jason
Technical SEO | | Jason_Marsh1230 -
Poor Site Performance
Hello, A couple of months ago, this site was dropped from google due to a noindex, nofollow tag thewealthymind(dot)com It's back up, but performing poorly. Take for example the term "The 4 step belief change" in the home page title tag. This site is the #1 authority on that and yet it ranks 3rd below weaker pages. There's 180 404 errors in GWT, many from past versions of pages of the site but also including thewealthymind(dot)com/index.html and thewealthymind(dot)com/index.htm even though there is a rel=cononical tag on the home page. What's the process of getting this site back to health?
Technical SEO | | BobGW0 -
Cross links between sites
hi, We have several ecommerce sites and we cross linked 3 of them by mistake. We realize that the sites were linked through WMT, We have shut down 2 of the sites about 2 months ago, but WMT still shows the links coming from those 2 sites. how do we make sure that google will see the sites are shut down. Is there a better of way resolving this issue. We are no longer using those sites, so do not need them to be active. whats the best solution to show google that the links are no longer there. Crawler shows that it was able to crawl the site 45 days after it is shut down. thanks nick
Technical SEO | | orion680 -
Young site trying hard, but banging head against the wall -- Site Review
Hi All New to PRO but we're seriously committed to getting this working. And firstly thank you to anyone who offers any useful thoughts and insights. We've launched a new site, unfortunately late to the market for the season and are really struggling to get search engine recognition. Site: http://www.ignitehats.co.uk/ We're continuously adding new content, slowly gathering more links and working hard to promote socially. But even on our clearest search terms like "Ignite hats" we're down on page 4. Both GWT and the Seomoz tools highlight no big problems (a few titles that are too long) but otherwise nothing. Maybe wrongly we requested that the Google spam team review our site incase it was being penalised, but got a template response saying the site was not in their spam system (phew, there wasn't a reason it should be we believe). We're wondering if this is just that our site is just too young? It's been live for 6 weeks. But worry maybe this is not the case. We've had success with another site we run much sooner than this. Any help or pointers would be really appreciated. Similar stories and what others have done, at least to give us some confidence to carry on would be great. Thanks for reading.
Technical SEO | | JHill0 -
Video submission sites
Hello, What are the top 5 sites for video submissions ? Any suggestions about which points should be taken into consideration when submitting videos ? Thanks
Technical SEO | | seoug_20050 -
Internal Links not Crawled by Open Site Explorer
Can someone plz tell me why www.hotelelgreco.gr has only 2 internal links in OSE despite the fact that the text content has a plethora of them. Thanks in advance.
Technical SEO | | socrateskirtsios0 -
Site forwarding - seo friendly or not?
Recently i decided to change my domain name - and although i have written several useful and working .htacess files with 301 redirects, this one became more complicated by the fact that I went through TWO domain name changes, before settling on the second one. Having seen some issues with the browser not being able to interpret correctly the .htaccess file, i temporarily suspended the .htaccess file, and opted instead for site forwarding. I don't know the mechanics behind site forwarding, or whether it is seo friendly or just a method for ip addressing, a sort of pseudo domain name server record change.
Technical SEO | | highersourcesites
I let it lie for a few weeks, until the dust settled, and yesterday put back the basic .htaccess file, with a 301 redirect, which directs the original domain name to be forwarded to the new one ( also it has a conditional in place to solve canonical issues). It works fine. But right now i am not seeing the link juice, the domain age, the domain page rank that it has. It has gone to zero, when it used to be three, sometimes four. I also made the change of address using webmaster tools. How long ( forever?) will it take to see my old page rank come back, even if it loses 10% from the change? And does site forwarding help or hinder seo ranking?0