Help! Pages not being indexed
-
Hi Mozzers, I need your help.
Our website (www.barnettcapitaladvisors.com) stopped being indexed in search engines following a round of major changes to URLs and content. There were a number of dead links for a few days before 301 redirects were properly put in place. And now, only 3 pages show up in bing when I do the search "site:barnettcapitaladvisors.com". A bunch of pages show up in Google for that search, but they're not any of the pages we want to show up.Our home page and most important services pages are nowhere in search results.
What's going on here?
Our sitemap is at http://www.barnettcapitaladvisors.com/sites/default/files/users/AndrewCarrillo/sitemap/sitemap.xml
Robots.txt is at: http://www.barnettcapitaladvisors.com/robots.txtThanks!
-
Thanks for the help! The Screaming Frog site crawler really makes it much easier to understand the robots.txt file. It looks like it was set to "disallow" robots.
-
Nothing really to "resolve" this.
those tools could simply provide you with cues and signs to figure out what went wrong and when... traffic and referral data on those pages could also indicate when they exactly stopped appearing in search and you can the correlate that with your dev actions if you do have some memory or log of those steps taken and backtrack and find out what was set wrong.
So you see absolutely no errors no messages nothing in GWT per this old post?
if you are using drupal, check this thread about it, it may well be your problem...
also see the very bottom of this page that may help you diagnose if you have a setting wrong in your robots file.
-
Hi Brien
The X-Robots tag has set many of your pages to noindex,nofollow. X-Robots is delivered as part of the HTTP response header so you won't find it in your HTML code.
In addition to that, your canonical tags are a little confused: the homepage canonical tag is set to
http://barnettcapitaladvisors.com/miami-financial-advisor-south-florida-financial-planner
when it should be barnettcapitaladvisors.com
I found these things by using Screaming Frog site crawler.
In your Robots file you're not allowing Yandex or Baidu to crawl the site - are you aware of this?
Have your developer correct the X-Robots and canonical issues.
-
Yes, I've searched Google Webmaster Tools and see a big dropoff in impressions on 5/4/13.
There are 14 pages in the sitemap that should be showing up. But, instead, what's getting indexed (9 pages in Google and 2 of the 3 pages in Bing) are pages that aren't in the sitemap -- ones that were created and then taken down or that we don't care to be indexed.
Yes, some of the URLs are new (three weeks ago) and that may be why they haven't appeared yet, but what about the home page? That isn't a new URL and it isn't appearing. Also, one new URL (which is also one of the important services pages that we want to appear) is appearing in bing: www.barnettcapitaladvisors.com/wealth-management but the other ones are not.I use analytics as well, but what particularly in Google Analytics will help me resolve this?
-
have you checked your google/bing webmasters tool yet? that often will give you the best answer in these situations?
Do you use analytics? Have you checked that as well?
if any pages do show up when you site search your domain on google, then you are being indexed, if pages you made are too new, it may take some time for them to become available in google's SERPs
it all just depends, how long has it been?
I see only 2 links in bing results for your site, but for google we have 9 results, how many should it be? 14?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Delete indexed spam pages
Hi everyone, I'm hoping someone had this same situation, or may know of a solution. One of our sites was recently pharmahacked 😞 We found an entire pharmaceutical site in one of the folder of our site. We were able to delete it, but now Google is showing us on not found error for those pages we deleted. First, I guess the question is will this harm us? If so, anyway we can fix this? Obliviously we don't want to do a 303 redirect for spam pages. Thanks!
Technical SEO | | Bridge_Education_Group0 -
How to know which pages are indexed by Google?
So apparently we have some sites that are just duplicates of our original main site but aiming at different markets/cities. They have completely different urls but are the same content as our main site with different market/city changed. How do I know for sure which ones are indexed. I enter the url into Google and its not there. Even if I put in " around " it. Is there another way to query google for my site? Is there a website that will tell you which ones are indexed? This is probably a dumb question.
Technical SEO | | greenhornet770 -
Help! Same page with multiple urls. How is this handled?
I'm using DotNetNuke for many of our sites. DotNetNuke's home page can have multiple VALID URLs that go to the same home page. Example: http://aviation-sms.com http://www.aviation-sms.com http://aviation-sms.com/default.aspx http://www.aviation-sms.com/default.aspx and http://aviation-sms.com/aviationSMS.aspx http://www.aviation-sms.com/aviationSMS.aspx All the above URLs have the same content. In the page header tag, I have: Should I be doing something else? such as removing the "default.aspx"??? I have a blog also that has a boatload of pages. I tried this canonical approach, but I'm not sure SEO Moz likes it and the tool offers me little guidance on this issue.
Technical SEO | | manintights280 -
Splitting Page Authority with two URLs for the same page.
Hello guys, My website is currently holding two different URLs for the same page and I am under the impression such set up is dividing my Page Authority and Link Juice. We currently have the following page with both URLs below: www.wbresearch.com/soldiertechnologyusa/home.aspx
Technical SEO | | JoaoPdaCosta-WBR
www.wbresearch.com/soldiertechnologyusa/ Analysing the page authority and backlinks I identified that we are splitting the amount of backlinks (links from sites, social media and therefore authority). "/home.aspx"
PA: 67
Linking Root Domains: 52
Total Links: 272 "/"
PA: 64
Linking Root Domains: 29
Total Links: 128 I am under the impression that if the URLs were the same we would maximise our backlinks and therefore page authority. My Question: How can I fix this? Should I have a 301 redirect from the page "/" to the "/home.aspx" therefore passing the authority and link juice of “/” directly to “/homes.aspx”? Trying to gather thoughts and ideas on this, suggestions are much appreciated? Thanks!0 -
Pages not indexed by Google
We recently deleted all the nofollow values on our website. (2 weeks ago) The number of pages indexed by google is the same as before? Do you have explanations for this? website : www.probikeshop.fr
Technical SEO | | Probikeshop0 -
My pages says it has 16 errors, need help
My pages says it has 16 errors, and all of them are due to duplicate content. How do I fix this? I believe its only due to my meta tag description.
Technical SEO | | gaji0 -
Redesign an SEO-Disaster | Help with Redirects of Gray Hat Pages
Hi gang. I'm a new SEO and I'm currently working on the redesign of a website. I have just discovered a ton of hidden pages that are filled with duplicate content, basically reiterating the main keyword in a variety of different variations. Each page is titled with the variation on the keyword phrase and then has one paragraph of text very similar to the previous page, etc. Here is an example of one of the offensive pages (nice lookin' site, eh?): http://www.vasectomy-reversals.com/vasectomy_reversal_surgery.html The new site will not have any of these pages. I'm writing the 301 redirects now and want to redirect these offensive pages to the most relevant page on the new site. But, I'm afraid to redirect the offensive pages. Should I leave them alone, or can I have the former developer remove them? Help. Don't know how to handle these pages and their redirects. Thanks for your help! ~ Mills
Technical SEO | | Mills0 -
Some site pages are removed from Google Index
Hello, Some pages of my clients website are removed from Google Index. We were in top 10 position for some keywords but now I cannot find those pages neither in top 1000. Any idea what to do in order to get these pages back? thank you
Technical SEO | | besartbajrami0