To index or de-index internal search results pages?
-
Hi there.
My client uses a CMS/E-Commerce platform that is automatically set up to index every single internal search results page on search engines. This was supposedly built as an "SEO Friendly" feature in the sense that it creates hundreds of new indexed pages to send to search engines that reflect various terminology used by existing visitors of the site. In many cases, these pages have proven to outperform our optimized static pages, but there are multiple issues with them:
- The CMS does not allow us to add any static content to these pages, including titles, headers, metas, or copy on the page
- The query typed in by the site visitor always becomes part of the Title tag / Meta description on Google. If the customer's internal search query contains any less than ideal terminology that we wouldn't want other users to see, their phrasing is out there for the whole world to see, causing lots and lots of ugly terminology floating around on Google that we can't affect.
I am scared to do a blanket de-indexation of all /search/ results pages because we would lose the majority of our rankings and traffic in the short term, while trying to improve the ranks of our optimized static pages. The ideal is to really move up our static pages in Google's index, and when their performance is strong enough, to de-index all of the internal search results pages - but for some reason Google keeps choosing the internal search results page as the "better" page to rank for our targeted keywords.
Can anyone advise? Has anyone been in a similar situation?
Thanks!
-
What cms are you using? I have a module that I developed that is free for Prestashop if you are using it. What it does is lets you give your search pages a static name. You have to specifically define terms, so it does not automatically make a landing page for each term. It also lets you do title's, meta descriptions, and keywords for the pages. The way the code works might prove a useful framework for adapting it to what ever platform you are using. The git is at https://github.com/Prestashop-Modules/searchterm there is an explanation of how it works at http://blog.dh42.com/search-pages-landing-pages/
-
My only point to that is that you will need a way to let people search on your site. Since you are taking the old search URLs and redirecting them, you will not be able to use them for people to search your site moving forward.
You would have to create a new search option using a new folder/url for this so that the old one could be 301ed to the static pages.
-
Thank you so much for your helpful answer. I am truly hoping what you described is possible using this CMS, which often makes it impossible to implement complicated customizations.
I am a little confused by what you mean by "create a new /search/" section. Are you saying that all of my new static pages should also live in the /search/ folder? Then how would I de-index the existing low-value search results pages?
-
You have to do this in phases.
-
Find the high ranking pages in search results - figure out why they are ranking.
-
Build the new static pages with good quality content to replace them. Use the info from #1 to help
-
Create a new /search/ section - put that behind robots.txt etc.
-
Update navigation links, sitemaps, breadcrumbs etc to now point to the new static optimized pages.
-
301 redirect the old /search/ section to the new static pages
-
Activate the new search section, but be careful about linking to it etc.
Done.
What is probably happening is that you have links to the site search sections from all over your site. You are telling Google that this is the most important part of your site. Google then says "OK" and pays attention to them. Using the steps above you redirect Googles attention, pass along the link/rank equity and prevent the new search section from being found.
Good luck!
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Magento 1.9 SEO. I have product pages with identical On Page SEO score in the 90's. Some pull up Google page 1 some won't pull up at all. I am searching for the exact title on that page.
I have a website built on Magento 1.9. There are approximately 290,000 part numbers on the site. I am sampling Google SERP results. About 20% of the keywords show up on page 1 position 5 thru 10. 80% don't show up at all. When I do a MOZ page score I get high 80's to 90's. A page score of 89 on one part # may show up on page one, An identical page score on a different part # can't be found on Google. I am searching for the exact part # in the page title. Any thoughts on what may be going on? This seems to me like a Magento SEO issue.
Intermediate & Advanced SEO | | CTOPDS0 -
Webmaster Tools Not Indexing New Pages
Hi there Mozzers, Running into a small issue. After a homepage redesign (from a list of blog posts to a product page), it seems that blog posts are buried on the http://OrangeOctop.us/ site. The latest write-up on "how to beat real madrid in FIFA 15", http://orangeoctop.us/against-real-madrid-fifa-15/ , has yet to be indexed. It would normally take about a day naturally for pages to be indexed or instantly with a manual submission. I have gone into webmaster tools and manually submitted the page for crawls multiple times on multiple devices. Still not showing up in the search results. Can anybody advise?
Intermediate & Advanced SEO | | orangeoctop.us0 -
Noindex search pages?
Is it best to noindex search results pages, exclude them using robots.txt, or both?
Intermediate & Advanced SEO | | YairSpolter0 -
Search Results not Updating (Title, Description, and URL)
Issue: I recently discovered that my site was accessible by both HTTP and HTTPS. The site has used a rel canonical tag to point to the HTTP version. Google+ was pointing to HTTPS though. The title, description, and URL shown in the results for the homepage is HTTPS, other pages are HTTP, etc... Steps taken to Resolve: This week I did the following... 301'd all non-checkout pages to the HTTP version Switched Google+ URL to HTTP version and added new post with an HTTP link to the homepage. Used webmaster tools to recrawl and reindex the site Resubmitted XML Sitemap No luck... the site is still not updating... any advice would be greatly appreciated. Thanks all! Site is Here
Intermediate & Advanced SEO | | AhlerManagement0 -
Thousands of Web Pages Disappered from Google Index
The site is - http://shop.riversideexports.com We checked webmaster tools, nothing strange. Then we manually resubmitted using webmaster tools about a month ago. Now only seeing about 15 pages indexed. The rest of the sites on our network are heavily indexed and ranking really well. BUT the sites that are using a sub domain are not. Could this be a sub domain issue? If so, how? If not, what is causing this? Please advise. UPDATE: What we can also share is that the site was cleared twice in it's lifetime - all pages deleted and re-generated. The first two times we had full indexing - now this site hovers at 15 results in the index. We have many other sites in the network that have very similar attributes (such as redundant or empty meta) and none have behaved this way. The broader question is how to do we get the indexing back ?
Intermediate & Advanced SEO | | suredone0 -
Any downsides of (permanent)redirecting 404 pages to more generic pages(category page)
Hi, We have a site which is somewhat like e-bay, they have several categories and advertisements posted by customers/ client. These advertisements disappear over time and turn into 404 pages. We have the option to redirect the user to the corresponding category page, but we're afraid of any negative impact of this change. Are there any downsides, and is this really the best option we have? Thanks in advance!
Intermediate & Advanced SEO | | vhendriks0 -
De-indexed Link Directory
Howdy Guys, I'm currently working through our 4th reconsideration request and just have a couple of questions. Using Link Detox (www.linkresearchtools.com) new tool they have flagged up a 64 links that are Toxic and should be removed. After analysing them further alot / most of them are link directories that have now been de-indexed by Google. Do you think we should still ask for them to be removed or is this a pointless exercise as the links has already been removed because its been de-indexed. Would like your views on this guys.
Intermediate & Advanced SEO | | ScottBaxterWW0 -
Ranking with other pages not index
The site ranks on page 4-5 with other page like privacy, about us, term pages. I encounter this problem allot in the last weeks; this usually occurs after the page sits 1-2 months on page 1 for the terms. I'm thinking of to much use the same anchor as a primary issue. The sites in questions are 1-5 pages microniche sites. Any suggestions is appreciated. Thank You
Intermediate & Advanced SEO | | m3fan0