Home Pages of Several Websites are disappearing / reappearing in Google Index

[wethink]

Hi,

I periodically use the Google site command to confirm that our client's websites are fully indexed.

Over the past few months I have noticed a very strange phenomenon which is happening for a small subset of our client's websites... basically the home page keeps disappearing and reappearing in the Google index every few days.  This is isolated to a few of our client's websites and I have also noticed that it is happening for some of our client's competitor's websites (over which we have absolutely no control).

In the past I have been led to believe that the absence of the home page in the index could imply a penalty of some sort.  This does not seem to be the case since these sites continue to rank the same in various Google searches regardless of whether or not the home page is listed in the index.

Below are some examples of sites of our clients where the home page is currently not indexed - although they may be indexed by the time you read this and try it yourself.  Note that most of our clients are in Canada.

My questions are:

1. has anyone else experienced/noticed this?

2. any thoughts on whether this could imply some sort of penalty? or could it just be a bug in Google?

3. does Google offer a way to report stuff like this?

Note that we have been building websites for over 10 years so we have long been aware of issues like www vs. non-www, canonicalization, and meta content="noindex" (been there done that in 2005).  I could be wrong but I do not believe that the site would keep disappearing and reappearing if something like this was the issue. Please feel free to scrutinize the home pages to see if I have overlooked something obvious - I AM getting old.

site:dietrichlaw.ca  - this site has continually ranked in the top 3 for [kitchener personal injury lawyers] for many years.

site:burntucker.com - since we took over this site last year it has moved up to page 1 for [ottawa personal injury lawyers]

site:bolandhowe.com - #1 for [aurora personal injury lawyers]

site:imranlaw.ca - continually ranked in the top 3 for [mississauga immigration lawyers].

site:canadaenergy.ca - ranks #3 for [ontario hydro plans]

Thanks in advance!

Jim Donovan, President

www.wethinksolutions.com

[BlueprintMarketing]

I just took the first domain you gave me I tested them on two tools you lack canonical's on all but the homepage for all three and all three failed the https://varvy.com test

1. imranlaw.ca
2. dietrichlaw.ca
3. canadaenergy.ca
4. burntucker.com  past the Varvy test but has only one canonical https://cl.ly/hPdN https://cl.ly/hPoe
5. bolandhowe.com  is the probably the most affected it has way too many 200 code URLs canonical's pointing to the HTTPS however they should be using a 301 redirect See  search engine land post below & these photos https://cl.ly/hPyM & https://cl.ly/hPUj

Preform a search and replace see: https://cl.ly/hPe6

• https://searchenginewatch.com/sew/how-to/2291162/seo-audit-findings-4-hidden-technical-problems-that-can-send-dangerous-signals-to-search-engines
• https://searchenginewatch.com/sew/how-to/2300520/technical-seo-for-nontechnical-people

I took the domainIn number three above and ran it through screaming frog I found no canonical's for all but one URL. Take a look at what most of the URLs appear like.

https://cl.ly/hPNg

In addition found that you have a redirect chain photos below they should go straight to HTTPS://www.canadaenergy.ca

I would utilize HSTS as well this will help considerably. And adding canonical's

https://cl.ly/hPJd to https://cl.ly/hPr1 to https://cl.ly/hPyj

Domain number two

• dietrichlaw.ca

the same situation you have one canonical URL homepage nothing else has a canonical

• https://cl.ly/hPzk
• https://cl.ly/hPlk

domain number one imranlaw.ca  same situation see below no canonical except for the homepage

| Address | http://www.imranlaw.ca/ |
| URL Encoded Address | http://www.imranlaw.ca/ |
| Status Code | 200 |
| Status | OK |
| Content | text/html; charset=ISO-8859-1 |
| Size | 13160 |
| Title 1 | Mississauga Immigration Lawyer & Canadian Citizenship Attorney |
| Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Mississauga Immigration Lawyer |
| H1-1 | Canadian Immigration & Naturalization Lawyer |
| H2-1 | Imran Khan Law Office offers Legal Services in Immigration Law and Real Estate Law Matters. |
| Meta Robots 1 | index,follow |
| Canonical Link Element 1 | http://www.imranlaw.ca/ |
| Word Count | 275 |
| Level | 1 |
| Inlinks | 28 |
| Outlinks |

19

|

| Address | http://www.imranlaw.ca/contact |
| URL Encoded Address | http://www.imranlaw.ca/contact |
| Status Code | 200 |
| Status | OK |
| Content | text/html; charset=ISO-8859-1 |
| Size | 14503 |
| Title 1 | Mississauga Immigration Lawyer - Contact |
| Meta Description 1 | Imran Khan - Canada Immigration lawyer and Canadian Citizenship attorney Contact Imran Khan |
| H1-1 | Contact Imran Khan Law Office |
| Meta Robots 1 | index,follow |
| Word Count | 276 |
| Level | 2 |
| Inlinks | 28 |
| Outlinks | 17 |

• https://cl.ly/hPsc
• https://cl.ly/hPdC
• https://cl.ly/hQ1I

A few domains the ones above which are listed below as well fail to be able to be seen by a synthetic Googlebot. Are you running them all on the same server?

You have some domains and in .com and others that end in .ca if you are looking in Google.ca and have geo-targeted the .com domains to Canada you should see them there. However if you're looking in Google.com obviously you cannot geo-target .CA domains to the United States therefore they would not show up in .com unless very rarely.

Deep crawl and screaming frog are going to be a best friends on this one. Please let me know if I can be of more help

here are my findings using a basic tool

dietrichlaw.ca

and put it into https://varvy.com

The results were

Findable links

Well formed static links not found.

Page has no findable links.

Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'

Learn about links and site hierarchy

HTTP headers

Page headers when accessed as Googlebot.

Headers:

Intro to HTTP

pages could not be found

https://varvy.com/hierarchyandlinks.html

Same thing for imranlaw.ca

Findable links

Well formed static links not found.

Page has no findable links.

Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'

Learn about links and site hierarchy

For canadaenergy.ca

Findable links

Well formed static links not found.

Page has no findable links.

Guideline states: 'Ensure that all pages on the site can be reached by a link from another findable page.'

Learn about links and site hierarchy

Amount of links

Amount of links not excessive.

0 links found on page.

Guideline states: 'Limit the number of links on a page to a reasonable number (a few thousand at most).'

Considering the amount of links on a page

**I wouldUse a tool like deepcrawl.com  or screamingfrog.co.uk/seospider **

two determined exactly what is wrong with all three Domains which failed a very basic test of being able to be detected by Googlebot.

Hope this helps,

Tom

[Andy.Drinkwater]

Hi Jim,

If analytics confirms that traffic is still landing on the homepage, then I think this is just Google reporting different pages when you perform a site: - It certainly doesn't sound like a penalty of any sort.

It is worth noting that Google did confirm some time back that site: doesn't bring back every page every time and is best used as a guide. Does the sitemap in Search Console show a healthy number of indexed links?

If you want a discussion on this, then it would be worthwhile also posting over at the Websearch Help Forums at Google and see what others have to say about it.

I hope this helps a little.

-Andy