Difficulty with Indexing Pages - Desperate for Help!
-
I have a website with product pages that use the same URL, but load different data based on what's passed to them with GET. I am using a Wordpress website, but all of the page information is retrieved from a database using PHP and displayed with PHP.
Somehow these pages are not being indexed by Google.
I have done the following:
1. Created a site map pointing to each page.
2. Defined URL parameters in Search Console for these type of pages.
3. Created a product schema using schema.org, and tested it without errors.
I have requested re-indexing repeatedly and these pages and images on the pages are still not being indexed! Does anybody have any suggestions?
-
Hey, David Butler . . . you are probably the only person who has looked at this issue, so I want to thank you again for your input.
I wanted to follow up about this. Since I am using Wordpress for the website, I used Wordpress's rewrite_rule function.
After confirming that it was working correctly, I deleted the old sitemaps and created new ones, then submitted them to Google.
Shortly after this, the pages were successfully indexed.
So from this experience I get the impression that using URL parameters doesn't necessary work -- I certainly am not going to use them in the future.
I am going to mark this issue as resolved.
-
Hi jacleaves,
I'm interested to see how this works out for you!
Please post an update after you've finished your work
Cheers,
David
-
Hey, David Butler,
I am at a loss when it comes to canonical tags -- I am trying to understand how to use them. However, in this case I thought it was set up correctly.
To better explain the website . . . all of the product pages use the same URL, which is http://amishdirectplaysets.com/playset-details/.
I programmed the pages to retrieve info from a database and display specific product information based on the parameters passed to them (?mfgID=A-1&catID=4). I cannot change the URL's to be anything different, as they are not actually different pages.
What I had read is that the canonical url should not include the URL parameters, so that is why they were just going to the same page. I created URL parameters in Google Search Console because I thought that would take care of this issue. However it is not.
So . . . what I have done (after reading your post) is gone ahead and changed the canonical URL to include the URL parameters.
Is that how I'm supposed to handle it in this type of a situation?
I am updating this response . . . after further investigation I can take care of changing the URL to something more search-engine friendly by using mod rewrite. I'll test it out.
-
Hi jacleaves,
The problem here is with your canonical tags (more info on canonical tags here).
This page: http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
Has a canonical tag pointing to: http://amishdirectplaysets.com/playset-details/
The same thing is happening on all of your product pages.
I would change all of your product pages to use "SEO-friendly" URLs and have a self-referring canonical tag.
Eg. your example page: http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
Would become: http://amishdirectplaysets.com/playset-details/a-1-standard-climber-vinyl-playset
And the canonical tag would point to: http://amishdirectplaysets.com/playset-details/a-1-standard-climber-vinyl-playsetCheers,
David
-
Hi Matthew,
Thanks for your quick response!
Here's one page:
http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
My issue is none of the individual product pages are being indexed.
In response to your starting suggestions:
1. I am not blocking any resources.
2. I am not using noindex.
3. Page is being rendered successfully.
One more thing . . . I mis-spoke about how the content is generated. The content on this page is generated by PHP and NOT Javascript. So that is a non-issue. (I corrected this in my initial question.)
I appreciate your feedback. Please let me know if you are able to determine what else I could be missing!
-
Hi jacleaves!
Without your website URL it will be difficult to diagnose the problem as there are a few potential technical issues here. My main worries would be whether you're blocking indexation somehow and the fact you're displaying your content using JavaScript.
I'd start here:
- Check you're not blocking crawlers in your robots.txt file
- Check you're not using the noindex tag
I'd also try using fetch and render in Google Search Console (https://support.google.com/webmasters/answer/6066468?hl=en) for your site pages to see how Google sees them. If googlebot isn't seeing your content, that's a strong hint there's a JavaScript crawling problem.
Also, strap yourself in and read this - https://www.elephate.com/blog/ultimate-guide-javascript-seo/ - it's THE guide for JavaScript SEO.
Let us know how you get on and we'll go from there!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google webcache of product page redirects back to product page
Hi all– I've legitimately never seen this before, in any circumstance. I just went to check the google webcache of a product page on our site (was just grabbing the last indexation date) and was immediately redirected away from google's cached version BACK to the site's standard product page. I ran a status check on the product page itself and it was 200, then ran a status check on the webcache version and sure enough, it registered as redirected. It looks like this is happening for ALL indexed product pages across the site (several thousand), and though organic traffic has not been affected it is starting to worry me a little bit. Has anyone ever encountered this situation before? Why would a google webcache possibly have any reason to redirect? Is there anything to be done on our side? Thanks as always for the help and opinions, y'all!
Intermediate & Advanced SEO | | TukTown1 -
May integrating my main category page in the index page improve my ranking of main category keyword?
90% of our sales are made with products in one of our product categories.
Intermediate & Advanced SEO | | lcourse
A search for main category keyword returns our root domain index page in google, not the category page.
I was wondering whether integrating the complete main category directly in the index page of the root domain and this way including much more relevant content for this main category keyword may have a positive impact on our google ranking for the main category keyword. Any thoughts?1 -
301 migration - Indexed Pages rising on old site
Hello, We did a 301 redirect from site a to site b back in March. I would check on a daily basis on the index count using query "site:sitename" The past couple of days, the old domain (that was 301 redirected) indexed pages has been rising which is really concerning. We did a 301 redirect back in march 2016, and the indexed count went from 400k pages down to 78k. However, the past 3 days it went from 78k to 89,500. And I'm worried that the number is going to continue to rise. My question - What would you do to investigate / how to investigate this issue? Would it be screaming frog and look at redirects? Or is this a unique scenario that I'd have to do other steps/procedures?
Intermediate & Advanced SEO | | ggpaul5620 -
Cache and index page of Mobile site
Hi, I want to check cache and index page of mobile site. I am checking it on mobile phone but it is showing the cache version of desktop. So anybody can tell me the way(tool, online tool etc.) to check mobile site index and cache page.
Intermediate & Advanced SEO | | vivekrathore0 -
Website Indexing Issues - Search Bots will only crawl Homepage of Website, Help!
Hello Moz World, I am stuck on a problem, and wanted to get some insight. When I attempt to use Screaming Spider or SEO Powersuite, the software is only crawling the homepage of my website. I have 17 pages associated with the main domain i.e. example.com/home, example.com/sevices, etc. I've done a bit of investigating, and I have found that my client's website does not have Robot.txt file or a site map. However, under Google Search Console, all of my client's website pages have been indexed. My questions, Why is my software not crawling all of the pages associated with the website? If I integrate a Robot.txt file & sitemap will that resolve the issue? Thanks ahead of time for all of the great responses. B/R Will H.
Intermediate & Advanced SEO | | MarketingChimp100 -
What is happening with this page's rankings? (G Analytics screenprint attached) help me.
Hi, At the moment im confused. I have a page which shows up for the query 'bank holidays' first page solid for 2 years - this also applies to the terms 'mothers day', 'pancake day' and a few others (UK Google). And there still ranking. Here is the problem: Usually I would rank for 'bank holidays 2014' (the terms with the year in are the real traffic drivers) and would be position 3/5. Over the last 3 months this has decayed dropping position to 30+. From the screenprint you can see the term 'Bank Holidays' is holding on but the term 'bank holidays 2014' is slowly decaying. If you query 'bank holidays 2015' we don't appear in rankings at all. What is causing this? The content is ok, social sharing happens and the odd link is picked up hear and there. I need help, how do I start pushing this back in the other direction, its like the site is slowly dying. And what really kills me, is 2 pages are ranking on page1 off link farms. URL: followuk.co.uk/bank-holidays serp-decay.jpg
Intermediate & Advanced SEO | | followuk0 -
How to find all indexed pages in Google?
Hi, We have an ecommerce site with around 4000 real pages. But our index count is at 47,000 pages in Google Webmaster Tools. How can I get a list of all pages indexed of our domain? trying to locate the duplicate content. Doing a "site:www.mydomain.com" only returns up to 676 results... Any ideas? Thanks, Ben
Intermediate & Advanced SEO | | bjs20100 -
Sitemap not indexing pages
My website has about 5000 pages submitted in the sitemap but only 900 being indexed. When I checked Google Webmaster Tools about a week ago 4500 pages were being indexed. Any suggestions about what happened or how to fix it? Thanks!
Intermediate & Advanced SEO | | theLotter0