Google & Bing not indexing a Joomla Site properly....
-
Can someone explain the following to me please.
The background:
I launched a new website - new domain with no history. I added the domain to my Bing webmaster tools account, verified the domain and submitted the XML sitemap at the same time. I added the domain to my Google analytics account and link webmaster tools and verified the domain - I was NOT asked to submit the sitemap or anything. The site has only 10 pages.
The situation:
The site shows up in bing when I search using site:www.domain.com - Pages indexed:- 1 (the home page) The site shows up in google when I search using site:www.domain.com - Pages indexed:- 30 Please note Google found 30 pages - the sitemap and site only has 10 pages - I have found out due to the way the site has been built that there are "hidden" pages i.e. A page displaying half of a page as it is made up using element in Joomla.
My questions:-
1. Why does Bing find 1 page and Google find 30 - surely Bing should at least find the 10 pages of the site as it has the sitemap? (I suspect I know the answer but I want other peoples input).
2. Why does Google find these hidden elements - Whats the best way to sort this - controllnig the htaccess or robots.txt OR have the programmer look into how Joomla works more to stop this happening.
3. Any Joomla experts out there had the same experience with "hidden" pages showing when you type site:www.domain.com into Google.
I will look forward to your input!
-
Thanks Ryan -
1. I thought as much with Bing but wanted to see other people thoughts - I will hunt around for the submit in webmaster tools. It begs the obvious question what's better quality (bing being selective) or quantity (google analysing it all and deciding for its self).... To be debated at length! lol
2 & 3. W3C no errors
and no css errors either..... I think it is the way we put the pages together using modules and laying them out via css - we employ our own coder. I don't really want to broadcast clients sites on forums etc.... But I am looking to improve to ensure we are doing things right - if something is not right we need to do it again and get it right. I don't want to get a rep for bad quality and bad work.
-
** Why does Bing find 1 page and Google find 30 **
Bing is much more selective then Google when it comes to indexing a site. Additionally, Bing takes longer as well. That has always been my experience but if others feel differently feel free to share.
Bing does has a way for you to manually submit all 10 pages. From the Bing Dashboard choose CONFIGURE > Submit URL, then enter each URL. By submitting the URL in this manner you can be certain Bing sees all your site's pages.
To be clear, Bing may crawl the page and choose not to index it. Bing also many index a page then later choose to drop it from their index. Bing has high quality standards related to content and various trust factors.
Why does Google find these hidden elements - Whats the best way to sort this - controllnig the htaccess or robots.txt OR have the programmer look into how Joomla works more to stop this happening.
Who built your site? Did you have a "random" developer build it? Or a professional Joomla developer who focuses only on building Joomla sites? How much experience does your developer have with the particular version of Joomla being used (likely 2.5 or 3.0)? Since you did not share your URL, the best I can offer is general advice. Try going using the HTML code validator from W3C. If you see dozens of errors then the site was not cleanly coded and you may have various issues.
I generally do not advice using robots.txt to block elements as they may still be crawled. I would need to view the site to offer more targeted advice.
Any Joomla experts out there had the same experience with "hidden" pages showing when you type site:www.domain.com into Google.
It can easily happen and typically occurs when a developer's focus is delivering the site rather then SEO. A developer's focus is typically satisfying you, their client, which is not unreasonable. Your requests likely focused on the appearance of the site and it's main functionality. It takes a lot more time and effort to developer an SEO optimized site when compared to a "regular" site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site move-Redirecting and Indexing dynamic pages
I have an interesting problem I would like to pick someone else’s brain. Our business has over 80 different products, each with a dedicated page (specs, gallery, copy etc.) on the main website. Main site itself, is used for presentation purpose only and doesn’t offer a direct path to purchase. A few years ago, to serve a specific customer segment, we have created a site where customers can perform a quick purchase via one of our major strategic partners. Now we are looking to migrate this old legacy service, site and all its pages under the new umbrella (main domain/CMS). Problem #1 Redirects/ relevancy/ SEO equity Ideally, we could simply perform 1:1 - 301 redirect from old legacy product pages to the relevant new site products pages. The problem is that Call to action (buy), some images and in some cases, parts of the copy must be changed to some degree to accommodate this segment. The second problem is in our dev and creative team. There are not enough resources to dedicate for the creation of the new pages so we can perform 1:1 301 redirects. So, the potential decision is to redirect a visitor to the dynamic page URL where parent product page will be used to apply personalization rules and a new page with dynamic content (buy button, different gallery etc.) is displayed to the user (see attached diagram). If we redirect directly to parent URL and then apply personalization rules, URL will stay the same and this is what we are trying to avoid (we must mention in the URL that user is on purchase path, otherwise this redirect and page where the user lands, can be seen as deceptive). Also Dynamic pages will have static URLs and unique page/title tag and meta description. Problem #2 : Indexation/Canonicalization The dynamic page is canonicalized to the parent page and does have nearly identical content/look and feel, but both serve a different purpose and we want both indexed in search. Hope my explanation is clear and someone can chip in. Any input is greatly appreciated! vCm2Dt.jpg
Intermediate & Advanced SEO | | bgvsiteadmin1 -
Why did Google cache & index a different domain than my own?
We own www.homemenorca.com, a real estate website based in Spain. Pages from this domain are not being indexed: https://www.google.com/search?q=site%3Awww.homemenorca.com&oq=site%3Awww.homemenorca.com&aqs=chrome..69i57j69i58j69i59l2.3504j0j7&sourceid=chrome&ie=UTF-8Please notice that the URLs are Home Menorca, but the titles are not Home Menorca, they are Fincas Mantolan, a completely different domain and company: http://www.fincasmantolan.com/. Furthermore, when we look at Google's cache of Home Menorca, we see a different website: http://webcache.googleusercontent.com/search?q=cache%3Awww.homemenorca.com%2Fen&oq=cache%3Awww.homemenorca.com%2Fen&aqs=chrome..69i57j69i58j69i59.1311j0j4&sourceid=chrome&ie=UTF-8We reviewed Google Search Console, Google Fetch, the canonical tags, the XML sitemap, and many more items. Google Search Console accepted our XML sitemap, but is only indexing 5-10% of the pages. Google is fetching and rendering the pages properly. However, we are not seeing the correct content being indexed in Google. We have seen issues with page loading times, loading content longer than 4 seconds, but are unsure why Google would be indexing a different domain.If you have suggestions or thoughts, we would very much appreciate it.Additional Language Issue:When a user searches "Home Menorca" from America or the UK with "English" selected in their browser as their default language, they are given a Spanish result. It seems to have accurate hreflang annotations within the head section on the HTML pages, but it is not working properly. Furthermore, Fincas Mantolan's search result is listed immediately below Home Menorca's Spanish result. We believe that if we fix the issue above, we will also fix the language issue. Please let us know any thoughts or recommendations that can help us. Thank you very much!
Intermediate & Advanced SEO | | CassG12340 -
Keyword On Page 1 Everywhere but Google (Site Specific)
Website: www.wheelchairparts.com
Intermediate & Advanced SEO | | Mike.Bean
Keyword: wheelchair parts My website is #1 or #2 on almost every search engine besides Google. Google has us bouncing between the bottom of page 2 and top of 3. However we are on page one for "wheelchairparts". I need to get a link building campaign going for this site. I feel it's more difficult for ecommerce websites and nothing seems to fit in with Rand's Mozcon 2016 Link Building talk except hacks. I need to find a flywheel. Either way, my question is what can I do other than link building to get on page 1 of Google for the term "wheelchair parts"? Thanks in advance! - Mike Bean1 -
Fix Google Index error
I changed my blog URL structure Can Someone please let me how to solve this?
Intermediate & Advanced SEO | | Michael.Leonard0 -
Ranking on google but not Bing?
Any reason why I could be ranking for Google but not Bing?
Intermediate & Advanced SEO | | edward-may0 -
Why are bit.ly links being indexed and ranked by Google?
I did a quick search for "site:bit.ly" and it returns more than 10 million results. Given that bit.ly links are 301 redirects, why are they being indexed in Google and ranked according to their destination? I'm working on a similar project to bit.ly and I want to make sure I don't run into the same problem.
Intermediate & Advanced SEO | | JDatSB1 -
How to resubmit a Web 2.0 site to Google?
I have 3 web 2.0 sites that look like theyve been hit by a penalty. I have checked their backlinks and there are a lot of backlinks from sites that have been deindexed. I have requested the removal of lots of the links, but now I need to resubmit the site to Google. Is this even possible with them being a web 2.0 site? I don't have webmaster tools for the site so how would I do this?
Intermediate & Advanced SEO | | JohnPeters0 -
Sites banned from Google?
How do you find out sites banned from Google? I know how to find out sites no longer cached, or is it the same thing once deindexed? As always aprpeciate your advice everyone.
Intermediate & Advanced SEO | | pauledwards0