Google & Bing not indexing a Joomla Site properly....
-
Can someone explain the following to me please.
The background:
I launched a new website - new domain with no history. I added the domain to my Bing webmaster tools account, verified the domain and submitted the XML sitemap at the same time. I added the domain to my Google analytics account and link webmaster tools and verified the domain - I was NOT asked to submit the sitemap or anything. The site has only 10 pages.
The situation:
The site shows up in bing when I search using site:www.domain.com - Pages indexed:- 1 (the home page) The site shows up in google when I search using site:www.domain.com - Pages indexed:- 30 Please note Google found 30 pages - the sitemap and site only has 10 pages - I have found out due to the way the site has been built that there are "hidden" pages i.e. A page displaying half of a page as it is made up using element in Joomla.
My questions:-
1. Why does Bing find 1 page and Google find 30 - surely Bing should at least find the 10 pages of the site as it has the sitemap? (I suspect I know the answer but I want other peoples input).
2. Why does Google find these hidden elements - Whats the best way to sort this - controllnig the htaccess or robots.txt OR have the programmer look into how Joomla works more to stop this happening.
3. Any Joomla experts out there had the same experience with "hidden" pages showing when you type site:www.domain.com into Google.
I will look forward to your input!
-
Thanks Ryan -
1. I thought as much with Bing but wanted to see other people thoughts - I will hunt around for the submit in webmaster tools. It begs the obvious question what's better quality (bing being selective) or quantity (google analysing it all and deciding for its self).... To be debated at length! lol
2 & 3. W3C no errors
and no css errors either..... I think it is the way we put the pages together using modules and laying them out via css - we employ our own coder. I don't really want to broadcast clients sites on forums etc.... But I am looking to improve to ensure we are doing things right - if something is not right we need to do it again and get it right. I don't want to get a rep for bad quality and bad work. -
** Why does Bing find 1 page and Google find 30 **
Bing is much more selective then Google when it comes to indexing a site. Additionally, Bing takes longer as well. That has always been my experience but if others feel differently feel free to share.
Bing does has a way for you to manually submit all 10 pages. From the Bing Dashboard choose CONFIGURE > Submit URL, then enter each URL. By submitting the URL in this manner you can be certain Bing sees all your site's pages.
To be clear, Bing may crawl the page and choose not to index it. Bing also many index a page then later choose to drop it from their index. Bing has high quality standards related to content and various trust factors.
Why does Google find these hidden elements - Whats the best way to sort this - controllnig the htaccess or robots.txt OR have the programmer look into how Joomla works more to stop this happening.
Who built your site? Did you have a "random" developer build it? Or a professional Joomla developer who focuses only on building Joomla sites? How much experience does your developer have with the particular version of Joomla being used (likely 2.5 or 3.0)? Since you did not share your URL, the best I can offer is general advice. Try going using the HTML code validator from W3C. If you see dozens of errors then the site was not cleanly coded and you may have various issues.
I generally do not advice using robots.txt to block elements as they may still be crawled. I would need to view the site to offer more targeted advice.
Any Joomla experts out there had the same experience with "hidden" pages showing when you type site:www.domain.com into Google.
It can easily happen and typically occurs when a developer's focus is delivering the site rather then SEO. A developer's focus is typically satisfying you, their client, which is not unreasonable. Your requests likely focused on the appearance of the site and it's main functionality. It takes a lot more time and effort to developer an SEO optimized site when compared to a "regular" site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap Indexed Pages, Google Glitch or Problem With Site?
Hello, I have a quick question about our Sitemap Web Pages Indexed status in Google Search Console. Because of the drastic drop I can't tell if this is a glitch or a serious issue. When you look at the attached image you can see that under Sitemaps Web Pages Indexed has dropped suddenly on 3/12/17 from 6029 to 540. Our Index status shows 7K+ indexed. Other than product updates/additions and homepage layout updates there have been no significant changes to this website. If it helps we are operating on the Volusion platform. Thanks for your help! -Ryan rou1zMs
Intermediate & Advanced SEO | | rrhansen0 -
HUGELY different ranking for a keyword between Bing and Google. Looking for ideas.
We rank really well for a brand in Bing (#2 behind manufacturer, and it's a competitive name) but are in about 15th place in Google. Any suggestions on what could be hurting us in Google are welcome!
Intermediate & Advanced SEO | | absoauto1 -
Product Pages not indexed by Google
We built a website for a jewelry company some years ago, and they've recently asked for a meeting and one of the points on the agenda will be why their products pages have not been indexed. Example: http://rocks.ie/details/Infinity-Ring/7170/ I've taken a look but I can't see anything obvious that is stopping pages like the above from being indexed. It has a an 'index, follow all' tag along with a canonical tag. Am I missing something obvious here or is there any clear reason why product pages are not being indexed at all by Google? Any advice would be greatly appreciated. Update I was told 'that each of the product pages on the full site have corresponding page on mobile. They are referred to each other via cannonical / alternate tags...could be an angle as to why product pages are not being indexed.'
Intermediate & Advanced SEO | | RobbieD910 -
Best way to remove full demo (staging server) website from Google index
I've recently taken over an in-house role at a property auction company, they have a main site on the top-level domain (TLD) and 400+ agency sub domains! company.com agency1.company.com agency2.company.com... I recently found that the web development team have a demo domain per site, which is found on a subdomain of the original domain - mirroring the site. The problem is that they have all been found and indexed by Google: demo.company.com demo.agency1.company.com demo.agency2.company.com... Obviously this is a problem as it is duplicate content and so on, so my question is... what is the best way to remove the demo domain / sub domains from Google's index? We are taking action to add a noindex tag into the header (of all pages) on the individual domains but this isn't going to get it removed any time soon! Or is it? I was also going to add a robots.txt file into the root of each domain, just as a precaution! Within this file I had intended to disallow all. The final course of action (which I'm holding off in the hope someone comes up with a better solution) is to add each demo domain / sub domain into Google Webmaster and remove the URLs individually. Or would it be better to go down the canonical route?
Intermediate & Advanced SEO | | iam-sold0 -
Spammy sites that link to a site
Hello, What is the best and quickest way to identify spammy sites that link to a website, and then remove them ( google disavow?) Thank you dear Moz, community - I appreciate your help 🙂 Sincerely, Vijay
Intermediate & Advanced SEO | | vijayvasu0 -
Will Google bots crawl tablet optimized pages of our site?
We are in the process of creating a tablet experience for a portion of our site. We haven’t yet decided if we will use a one URL structure for pages that will have a tablet experience or if we will create separate URLs that can only be access by tablet users. Either way, will the tablet versions of these pages/URLs be crawled by Google bots?
Intermediate & Advanced SEO | | kbbseo0 -
Google is not Indicating any Links to my site
We built a new store on another ccTLD and linked to it from some of our other domains in a few locations. I am noticing that with the Google operator command "links:" we are seeing nothing linking to our site anywhere. Some things to clarify: These are not no-follow links These pages linking to our new domain are indexed The pages being linked to on our new domain are indexed This is not a flash site or heavy in JavaScript The links existed the day the site was launched so when the new pages were crawled they existed. "Site:" command in Google shows me that my new site is indexed. What could potentially be causing this? I am trying to get these newer ccTLD's to begin ranking and I understand that I need to get links going to these pages since they are fairly new (2.5 months) so I can outrank the .com in the SE's in those locales. (Like Google.co.uk)
Intermediate & Advanced SEO | | DRSearchEngOpt0 -
Getting Google to Correct a Misspelled Site Link...Help!
My company website recently got its site links in google search... WooHoo! However, when you type TECHeGO into Google Search one of the links is spelled incorrectly. Instead of 'CONversion Optimization' its 'COversion Optimization'. At first I thought there was a misspelling on that page somewhere but there is not and have come to the conclusion that Google has made a mistake. I know that I can block the page in webmaster tools (No Thanks) but how in the crap can I get them to correct the spelling when no one really knows how to get them to appear in the first place? Riddle Me That Folks! sitelink.jpg
Intermediate & Advanced SEO | | TECHeGO0