Google search issue with exact domain
-
We had a site from Feb-2011 to Nov-2011 at the domain amcoexterminating.com. The site was pure HTML/CSS and the daily unique visitors steadily increased over that time. So all was fine.
We then moved the site to a CMS (Joomla) on Dec. 6th. From that day forward, the daily visitors went into the tank.
Before the move, if you typed "amcoexterminating.com" or "amco exterminating" into Google search, the site would be the first result (as you'd expect since those are the words that make up the actua domain). But we tried this yesterday and the site did not come up at all. NOT GOOD. It would work in Yahoo or Bing, but not in Google. So obviously, the problem with Google search directly affected the daily visitors.
We just checked Webmaster tools yesterday (yes, this should have been done sooner, lesson learned) and it said "Site has severe health issues - Important page blocked by robots.txt". It listed the "important" page URL and it was just a link to an image. Regardless, I wiped out the Joomla created robots.txt file and added a new one and made it just say...
User-agent: *Allow: /
About 14 hours later, after the new robots.txt file was recognized by Google, the "severe health" message went away. However if I search in Google for "amcoexterminating.com", it still doesn't show up and the client is concerned (as they should be).
Do you think the search engines just need more time to refresh? If so, once it refreshes, should the site show up first again right away?
Or is it possible the robots.txt file had nothing to do with the issue? If so, what other things could I check into that might cause Google search to not find a site even if you search for exact domain name?
Please share any and all things I should look into as I need to get this site showing in Google search again (as it was before moving to the CMS).
Thanks!
-
Sorry I never replied...It all looks good now.
-
Hi Rob,
How are things going on this site? Are your rankings back, or are you still looking for some help?
-
Yes thats all you needed to do.
it should come back, with ine a few weeks
-
Update: I found this code in my htaccess file...
Header set X-Robots-Tag "noindex, nofollow"
So I commented it out like so..
#Header set X-Robots-Tag "noindex, nofollow"
And resaved it.
I imagine this is all I needed to do. If you think I should have done something else, please let me know.
-
Thanks Robert, the answer to your two "Edit" questions is Yes (htaccess.txt was renamed to .htaccess and "seo friendly URLs" was selected as YES).
It makes sense that what Alan suspects is my main problem (i.e. there is a no-index, no-follow on the homepage) so I'm hoping he can point me in the right direction to correct it. If you know and see my reply before he does, please let me know
-
Thanks for the heads up on the canonical issues, will correct that.
As for this X-robots header, do you know where I can find that code to either change the no-index, no-follow to index, follow, or delete it completely? I tried to view what you said in IE, but when I click F12, the window that pops up doesn't have a "Networks" tab.
I am searching my site files for X-robots header and can't find this area, so if you could give me any insight on correcting this, I would be very appreciative. If that is the problem (and I trust that it is), I really gotta get it fixed ASAP
Side note you may not know: If I correct the X-robots header to be index, follow, should the site start showing up again in Google results immediately or could it take longer?
-
Rob, you have a problem. Or two.
First, somehow your images files are showing up and I believe this is causing a duplicate content issue. If I were to guess, I would say that you are new to joomla and may have set something up incorrectly. Without seeing your Joomla dashboard, it is hard to say exactly what is going on though.
When you do a Site:www.amcoexterminating.com or Site:amcoexterminating.com you get mostly images and pdfs. You also see excluded content and when you click on it you see the same.
The only site url that is not a pdf is http://www.amcoexterminating.com/images/ so Google is not seeing any other url. This is due to Robots.txt or bad site map, Joomla settings that control these, etc.
I wish I could be of more help, but without seeing what the settings are in the dashboard, I cannot find the issue (using firebug, etc.).
Best
Edit: Alan has some good info in his response. Also, in Joomla on the Global Configuration there is an SEO Settings near top on right. Within that is a selection for Use Apache mod_rewrite if you selected this did you rename htaccess.txt to .htaccess before you activated this?
Above it is a selection for search engine friendly urls, have you selected that? -
No I cant see one image blocked being the problem, you should not block images, js or css files that are used on pages that are not blocked, it is not a problem alone, but if there are any other signals of spam, then the fact that you hide these files is then seen as a suspicious signal.
I am not a fan of CMS, as they do not write clean code. I had a look at your site and found many problems. The worst is canonical issues, you home page rank is split between
http://www.amcoexterminating.com/index.php
http://www.amcoexterminating.com
http://amcoexterminating.com/index.php
http://amcoexterminating.comYou need to fix your canonical issues.
http://thatsit.com.au/seo/tutorials/how-to-fix-canonical-domain-name-issuesbut the reason you cant find your page in Google is you have a X-robots header of no-index, no-follow. You have a index, follow meta tag, but the header must be overruling it.
If you want to see this for yourself. in IE press F12, select the network tab and start capturing. Load your home page, then click on the home page url in the results and select response headers.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to Redirect Old Domain to a New Domain?
One of my clients has chosen a new domain for her website so we will redirect the old domain to the new. I understand that once redirected, most of the authority built up over the years for the old domain will pass through to the new domain. Is this correct? Can this be done through the Wordpress dashboard, say with a plugin perhaps? I have a plugin called 404 Redirected I was going to try to use for this purpose. Finally, the client has other domains (5 total) she has used for one reason or another. She wants to consolidate all of them and have everything redirected to her new domain. Do you see any downside to this in terms of crawling, Google trust and authority? I know there are some user experience issues to deal with, but we're addressing those. I just don't want to do anything that may cause Google not to trust the new domain. Thanks, Dino
Web Design | | Dino640 -
Links not visible in "Google cache text version" but visible in "Fetch as Google" in Webmaster tool
Hi Guys, There seems some issue with the coding due to which Google is not indexing half of our menu bar links. The cached text version of http://www.99acres.com/ is not showing links present in dropdown "All India" , dropdown "Advice" and "Hot Projects" tab in blue bar on top menu whereas these links are visible in "Fetch as Google" in Google Webmaster tool. Any clue to why is there a difference between the links shown in Google webmaster and Google cache text version. Thanks in advance 🙂
Web Design | | vivekrathore0 -
Site is getting crushed by spam traffic and Google Webmaster Tools giving crawl warnings. Also...
Currently hosting a site I'm planning on moving to a new server ASAP, 301 redirecting and have a domain that has nice authority and very old. On the current site I need to clean up the blog. I have a few questions actually.... 1. I'd like to remove most of the blog articles as I want the new site to be very high quality, but isn't it dangerous to do a 301 redirect to the same page for all these articles? 2. I want to focus on the new site as the current site has too many issues but still managing to hang in their. is highly outdated yet I don't want to spend a ton of time on the site before the 301 redirect. With the Pigeon and Panda 4.0 rumors being released soon, I want to get the new site completed ASAP. Do you think it's better if I fix the 3. Would removing cloudflare make things better or worse with the crashing of my site due to high traffic (mainly spam on the blog.) 4. My best article by far is outdated, but should I waste time updating it before redirecting or should I just get the new site going? I did way too many guest posts thinking content is king, but at least checked the outgoing links Domain Auth, Page Auth, and MozTrust in OSE, but first off I'm going to remove a page that mentions I'm looking for guest bloggers. I tried to keep the posts relevant but at the time you could get away with 5. Anything I can do to slow down these spammers on Wordpress? I noticed most of them are checking for vulnerabilities but I'm keeping it up to date, have caching setup. Thanks!
Web Design | | eugenecomputergeeks0 -
Getting a highly ranked site a better result for 1 search term
I have a highly ranked website for a niche category. My site ranks higher in SEOMOZ than all of my competitors, but I can't get any higher than 4th on a page for one specific search term. What can I do to help my site increase its ranking on a specific search term?
Web Design | | tadden0 -
Given the lastest Google update, should I rewrite my Flash site or try to present an alternative HTML/CSS site?
I have a site that was created using Flash. The reasoning behind this was, at the time, that I didn't care if the site ranked or not (portfolio site). Now I would like to drive traffic to the site from SE's. Given the Penguin update, should I rewrite my Flash site in HTML/CSS or present an alternative site for bots and browsers that don't support Flash? My concern is that by presenting an alternative site to bots and non Flash supporting browsers that the SE's will see potentially see this as cloaking. Thoughts and advice would be much appreciated.
Web Design | | mj7750 -
Why is this page removed from Google & Bing indices?
This page has been removed from indices at Bing and Google, and I can't figure out why. http://www.pingg.com/occasion/weddings This page used to be in those indices There are plenty of internal links to it The rest of the site is fine It's not blocked by meta robots, robots.txt or canonical URL There's nothing else to suggest that the page is being penalized
Web Design | | Ehren0 -
Has Anyone Had Issues With ASP.NET 4.0 URL Routing?
I'm seeing some odd results in my SEOMOZ results with a new site I just released that is using the ASP.NET 4.0 URL routing. I am seeing thousands(!) of duplicate results, for instance, because the crawl has uncovered something like this: http://www.mysite.com/
Web Design | | TroyCarlson
http://www.mysite.com/default.aspx (so far, so good, though I wish it wouldn't show both)
http://www.mysite.com/default.aspx/about/ (what the heck -?)
http://www.mysite.com/default.aspx/about/about/ (WTF!?)
http://www.mysite.com/default.aspx/about/about/products/ (and on and on ad infinitum) I'm also seeing problems pop up in my sitemap because extensionless urls have an odd "eurl.axd/abunchofnumbersgohere" appended to the end of every address which is breaking links. sigh Buyer beware. I've found articles that discuss the "eurl.axd" issue here and there (this one seems very good), but nothing about the weird crawl issue I outlined above. Any advice?0 -
Facing design issue in IE6 and IE7
Hi, I implemented custom search code on my website which is on a test url http://61.246.241.232/. It is working perfectly fine on all the browsers except on IE6 and IE7. Can anyone please let me know the solution to this problem. Thanks & Regards
Web Design | | IM_Learner0