Hello, I have been building websites for about 4 months now and finally had my first real success with a website.
I found a niche that I was able to get on the first page with. This site was fine and then boom it dropped to #17 around the time of the recent Google changes. So I just thought it was that, I decided to add it as a campaign on here.
This is when I noticed that the pages were not being crawled. So being new at all of this, I researched that. Of course, the main things were is the privacy and robots.txt. It is running on wordpress and I know not to set the blog to private, but I wasn't familiar with how to edit the robots.txt.
I found a good plugin that easily allowed me to set the text to allow all bots. It seemed to be set to the normal wordpress settings before, and I never had a problem with any of my websites not being crawled.
Anyway, once I just set it to allow all bots on ALL of my website, the pages started being crawled again.
My traffic went back up and I was on the first page all day yesterday.
Today, so far big drop off. So, I deleted the campaign and set up a new one. Sure enough no pages crawled yet.
I made some security changes using Bulletproof Security and another plugin to see if that effects it. Nothing yet.
I am just really confused as to what is going on, so if any of you have any ideas that would be great.
It is a simple site, and I made some changes like theme when I was trying to figure out why the pages weren't being crawled. So it is not the most beautiful design right now. Also, I try my best to put up well-written useful content, so I don't think that is the issue for the rankings drops.
I don't have many if any actual backlinks yet because of the newness of my site, could be the reason for it acting strange BUT none of that explains the pages not being crawled at the same time my site drops?????
Sorry so long but had to explain it all! Thanks in advance to anyone who has anything to say about this situation!
Edit: I should clarify, I put the the security plugins on yesterday while I was on the first page and have deleted them to see if that allows the pages to be crawled. Sorry if I wasn't clear.