Difficulty with Indexing Pages - Desperate for Help!
-
I have a website with product pages that use the same URL, but load different data based on what's passed to them with GET. I am using a Wordpress website, but all of the page information is retrieved from a database using PHP and displayed with PHP.
Somehow these pages are not being indexed by Google.
I have done the following:
1. Created a site map pointing to each page.
2. Defined URL parameters in Search Console for these type of pages.
3. Created a product schema using schema.org, and tested it without errors.
I have requested re-indexing repeatedly and these pages and images on the pages are still not being indexed! Does anybody have any suggestions?
-
Hey, David Butler . . . you are probably the only person who has looked at this issue, so I want to thank you again for your input.
I wanted to follow up about this. Since I am using Wordpress for the website, I used Wordpress's rewrite_rule function.
After confirming that it was working correctly, I deleted the old sitemaps and created new ones, then submitted them to Google.
Shortly after this, the pages were successfully indexed.
So from this experience I get the impression that using URL parameters doesn't necessary work -- I certainly am not going to use them in the future.
I am going to mark this issue as resolved.
-
Hi jacleaves,
I'm interested to see how this works out for you!
Please post an update after you've finished your work
Cheers,
David
-
Hey, David Butler,
I am at a loss when it comes to canonical tags -- I am trying to understand how to use them. However, in this case I thought it was set up correctly.
To better explain the website . . . all of the product pages use the same URL, which is http://amishdirectplaysets.com/playset-details/.
I programmed the pages to retrieve info from a database and display specific product information based on the parameters passed to them (?mfgID=A-1&catID=4). I cannot change the URL's to be anything different, as they are not actually different pages.
What I had read is that the canonical url should not include the URL parameters, so that is why they were just going to the same page. I created URL parameters in Google Search Console because I thought that would take care of this issue. However it is not.
So . . . what I have done (after reading your post) is gone ahead and changed the canonical URL to include the URL parameters.
Is that how I'm supposed to handle it in this type of a situation?
I am updating this response . . . after further investigation I can take care of changing the URL to something more search-engine friendly by using mod rewrite. I'll test it out.
-
Hi jacleaves,
The problem here is with your canonical tags (more info on canonical tags here).
This page: http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
Has a canonical tag pointing to: http://amishdirectplaysets.com/playset-details/
The same thing is happening on all of your product pages.
I would change all of your product pages to use "SEO-friendly" URLs and have a self-referring canonical tag.
Eg. your example page: http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
Would become: http://amishdirectplaysets.com/playset-details/a-1-standard-climber-vinyl-playset
And the canonical tag would point to: http://amishdirectplaysets.com/playset-details/a-1-standard-climber-vinyl-playsetCheers,
David
-
Hi Matthew,
Thanks for your quick response!
Here's one page:
http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
My issue is none of the individual product pages are being indexed.
In response to your starting suggestions:
1. I am not blocking any resources.
2. I am not using noindex.
3. Page is being rendered successfully.
One more thing . . . I mis-spoke about how the content is generated. The content on this page is generated by PHP and NOT Javascript. So that is a non-issue. (I corrected this in my initial question.)
I appreciate your feedback. Please let me know if you are able to determine what else I could be missing!
-
Hi jacleaves!
Without your website URL it will be difficult to diagnose the problem as there are a few potential technical issues here. My main worries would be whether you're blocking indexation somehow and the fact you're displaying your content using JavaScript.
I'd start here:
- Check you're not blocking crawlers in your robots.txt file
- Check you're not using the noindex tag
I'd also try using fetch and render in Google Search Console (https://support.google.com/webmasters/answer/6066468?hl=en) for your site pages to see how Google sees them. If googlebot isn't seeing your content, that's a strong hint there's a JavaScript crawling problem.
Also, strap yourself in and read this - https://www.elephate.com/blog/ultimate-guide-javascript-seo/ - it's THE guide for JavaScript SEO.
Let us know how you get on and we'll go from there!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Need some help understanding SEO - Please help before I lose [pull out] all my hair
I'm new to SEO, and am stubbornly trying to educate myself. I have a telescope shop in Canada, it's a small business that we run on the side. We're driving lots of traffic through FB and our outreach programs but I really want to increase our presence on search. We released a new website back in January and it killed some of our rankings. We're working our way back with a very specific set of efforts on regular SEO: Metadata and titles, although it seems that's not super relevant Building high quality backlinks and eliminating any spammy backlinks Rewriting product listings so that they are original content though I'm not sure how important this is in e-commerce Writing high quality articles and blog posts Working relevant keywords into our product pages and titles I understand that good SEO is about pushing on all the levers, and trying to make sure that your site is as valuable to the end user as possible. We're making some good progress, but I'm puzzled by the #1 shop in Canada. They don't put any apparent effort into SEO and they still rank #1 on every key product we compete with them on. I've worked with two separate, highly ranked and regarded SEO firms on this and neither has been able to tell my why this other site ranks so highly. Here's a specific example on a popular product that we both sell, the Celestron NexStar 8SE. Here’s the link to Telescope Canada’s page for their Celestron 8SE: https://telescopescanada.ca/products/celestron-nexstar-8se-computerized-telescope-11069 Here’s a link to the Celestron 8SE page from the manufacturer website: https://www.celestron.com/products/nexstar-8se-computerized-telescope Telescopes Canada has just copied and pasted. There is no original content aside from adding the shipping and return policy to the tab, and having some options for selecting accessories on the page. Here is our page: https://all-startelescope.com/products/celestron-nexstar-8se We have higher page authority, higher domain authority, and they keyword analyzer in moz says that our page is higher quality than the Telescopes Canada page. I can’t find a single metric on any tool (ubbersuggest, Moz, ahrefs, semrush) that says Telescopes Canada is a better site, or has a better NexStar 8SE product page. But they keep ranking ahead of us, and right at the top of google search. Our titles are good, our metadata is good (but I don’t think that’s been a serious ranking factor for about ten years). Our text is original, it’s relevant, we have healthy internal links to the page. According to Moz's page ranker it's 20 points higher than Telescope Canada's page. We have invensted in some excellent blog content, we’re adding new products to the website so that we rank for more keywords. All of those things are helping, but I fundamentally don’t understand why Telescopes Canada is #1 almost across the board on every key product in our market. There is something that I’m not seeing here. Can you see any metric, any tool in your toolbox that indicates why they rank at the top, or even higher than we do for in these search terms specific to that product: Celestron NexStar 8SE
Intermediate & Advanced SEO | | nkennett
NexStar 8SE
Celestron NexStar 8SE Canada
NexStar 8SE Canada I have a feeling it's something technical that I'm missing, but I'm not sure how obvious it is with two 'professional' firms not finding it. I'd really appreciate any help or insight that you can offer.0 -
Does redirecting a duplicate page NOT in Google‘s index pass link juice? (External links not showing in search console)
Hello! We have a powerful page that has been selected by Google as a duplicate page of another page on the site. The duplicate is not indexed by Google, and the referring domains pointing towards that page aren’t recognized by Google in the search console (when looking at the links report). My question is - if we 301 redirect the duplicate page towards the one that Google has selected as canonical, will the link juice be passed to the new page? Thanks!
Intermediate & Advanced SEO | | Lewald10 -
Need help in de-indexing URL parameters in my website.
Hi, Need some help.
Intermediate & Advanced SEO | | ImranZafar
So this is my website _https://www.memeraki.com/ _
If you hover over any of the products, there's a quick view option..that opens up a popup window of that product
That popup is triggered by this URL. _https://www.memeraki.com/products/never-alone?view=quick _
In the URL you can see the parameters "view=quick" which is infact responsible for the pop-up. The problem is that the google and even your Moz crawler is picking up this URL as a separate webpage, hence, resulting in crawl issues, like missing tags.
I've already used the webmaster tools to block the "view" parameter URLs in my website from indexing but it's not fixing the issue
Can someone please provide some insights as to how I can fix this?0 -
Why are some pages indexed but not cached by Google?
The question is simple but I don't understand the answer. I found a webpage that was linking to my personal site. The page was indexed in Google. However, there was no cache option and I received a 404 from Google when I tried using cache:www.thewebpage.com/link/. What exactly does this mean? Also, does it have any negative implication on the SEO value of the link that points to my personal website?
Intermediate & Advanced SEO | | mRELEVANCE0 -
Effect of Removing Footer Links In all Pages Except Home Page
Dear MOZ Community: In an effort to improve the user interface of our business website (a New York CIty commercial real estate agency) my designer eliminated a standardized footer containing links to about 20 pages. The new design maintains this footer on the home page, but all other pages (about 600 eliminate the footer). The new design does a very good job eliminating non essential items. Most of the changes remove or reduce the size of unnecessary design elements. The footer removal is the only change really effect the link structure. The new design is not launched yet. Hoping to receive some good advice from the MOZ community before proceeding My concern is that removing these links could have an adverse or unpredictable effect on ranking. Last Summer we launched a completely redesigned version of the site and our ranking collapsed for 3 months. However unlike the previous upgrade this modifications does not URL names, tags, text or any major element. Only major change is the footer removal. Some of the footer pages provide good (not critical) info for visitors. Note the footer will still appear on the home page but will be removed on the interior pages. Are we risking any detrimental ranking effect by removing this footer? Can we compensate by adding text links to these pages if the links from the footer are removed? Seems irregular to have a home page footer but no footer on the other pages. Are we inviting any downgrade, penalty, adverse SEO effect by implementing this? I very much like the new design but do not want to risk a fall in rank and traffic. Thanks for your input!!!
Intermediate & Advanced SEO | | Kingalan1
Alan0 -
More Indexed Pages than URLs on site.
According to webmaster tools, the number of pages indexed by Google on my site doubled yesterday (gone from 150K to 450K). Usually I would be jumping for joy but now I have more indexed pages than actual pages on my site. I have checked for duplicate URLs pointing to the same product page but can't see any, pagination in category pages doesn't seem to be indexed nor does parameterisation in URLs from advanced filtration. Using the site: operator we get a different result on google.com (450K) to google.co.uk (150K). Anyone got any ideas?
Intermediate & Advanced SEO | | DavidLenehan0 -
Does having multiple links to the same page influence the Link juice this page is able to pass
Say you have a page and it has 4 outgoing links to the same internal page. In the original Pagerank algo if these links were links to an page outside your own domain, this would mean that the linkjuice this page is able to pass would be devided by 4. The thing is i'm not sure if this is also the case when the outgoing link, is linking to a page on your own domain. I would say that outgoing links (whatever the destination) will use some of your link juice, so it would be better to have 1 outgoing link instead of 4 to the same destination, the the destination will profit more form that link. What are you're thoughts?
Intermediate & Advanced SEO | | TjeerdvZ0 -
Removing pages from index
Hello, I run an e-commerce website. I just realized that Google has "pagination" pages in the index which should not be there. In fact, I have no idea how they got there. For example, www.mydomain.com/category-name.asp?page=3434532
Intermediate & Advanced SEO | | AlexGop
There are hundreds of these pages in the index. There are no links to these pages on the website, so I am assuming someone is trying to ruin my rankings by linking to the pages that do not exist. The page content displays category information with no products. I realize that its a flaw in design, and I am working on fixing it (301 none existent pages). Meanwhile, I am not sure if I should request removal of these pages. If so, what is the best way to request bulk removal. Also, should I 301, 404 or 410 these pages? Any help would be appreciated. Thanks, Alex0