How to effectively de-index in Magento site?
-
We have thousands of Missing Description issues but most of them are account/login pages.
i.s. /customer/account/ etc...
We tried to de-index them through the Configuration using the instructions here - https://docs.magento.com/user-guide/marketing/search-engine-robots.html
But they're still appearing as issues in the Site Crawl.
Even without the site crawl issue, we don't really want these to appear in the SERPs.
Does anybody know how to properly de-index these login pages in Magento?
Thank you!
-
Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. Additionally, if you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws. -
To effectively de-index a Magento site, you can follow these steps:
Use the "Robots Meta Tag" to prevent indexing: You can add a meta tag to the header of your web pages to instruct search engines not to index them.
Use the "Robots.txt" file: You can use the robots.txt file to disallow search engine crawlers from accessing certain pages on your site.
Use the "Noindex" directive: Within the HTML code of your web pages, you can use the "noindex" directive to prevent search engines from indexing specific pages.
Use the "Canonical URL" tag: You can use the canonical URL tag to specify the preferred version of a web page, which can help prevent duplicate content from being indexed.
It's important to note that de-indexing pages should be done carefully, as it can impact your site's visibility in search engine results. If you have specific pages or sections in mind that you'd like to de-index, please let me know so I can provide more detailed guidance.
-
@LASClients Hey LASClients,
I feel your pain with those pesky login pages showing up in the Site Crawl. Have you considered using the Disallow directive in the robots.txt file to prevent search engines from crawling these pages? It's a quick fix, but as always, test it out in a staging environment first. Cheers!
Best,
[omaze cornwall] -
Certainly! To effectively de-index login pages in Magento and address the Missing Description issues, follow these steps:
Robots Meta Tag:
Open the respective login page templates, such as /customer/account/, in your Magento admin.
Add the following meta tag to the <head> section of the HTML:
html
<meta name="robots" content="noindex, nofollow">
This tag instructs search engines not to index the page and not to follow any links on it.
Robots.txt File:Edit your robots.txt file in the root of your Magento installation.
Add the following lines to disallow crawling of login pages: User-agent: *
Disallow: /customer/account/
Replace /customer/account/ with the relevant path for your login pages.
XML Sitemap:If you have an XML sitemap, ensure that the login pages are excluded from it.
Open your XML sitemap file and remove or comment out the entries related to login pages.
Submit Updated Sitemap to Search Engines:After making these changes, resubmit your updated XML sitemap to search engines via Google Search Console or Bing Webmaster Tools.
Clear Cache and Reindex:Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. If you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws. -
@get1200 @get1200
Certainly! To effectively de-index login pages in Magento and address the Missing Description issues, follow these steps:Robots Meta Tag:
Open the respective login page templates, such as /customer/account/, in your Magento admin.
Add the following meta tag to the <head> section of the HTML:
<meta name="robots" content="noindex, nofollow">
This tag instructs search engines not to index the page and not to follow any links on it.
Robots.txt File:Edit your robots.txt file in the root of your Magento installation.
Add the following lines to disallow crawling of login pages:
User-agent: *
Disallow: /customer/account/
Replace /customer/account/ with the relevant path for your login pages.
XML Sitemap:If you have an XML sitemap, ensure that the login pages are excluded from it.
Open your XML sitemap file and remove or comment out the entries related to login pages.
Submit Updated Sitemap to Search Engines:After making these changes, resubmit your updated XML sitemap to search engines via Google Search Console or Bing Webmaster Tools.
Clear Cache and Reindex:Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. Additionally, if you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws. -
@LASClients Create a file app/design/frontend/[Vendor]/[theme]/Magento_Customer/layout/customer_account_login.xml with the following content:
<?xml version="1.0"?> <page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd"> <head> <meta name="robots" content="noindex,nofollow" /> </head> </page>Clear cache
php bin/magento cache:flushAnd it should be fine.
-
@LASClients you could try adding the below meta in the pages that you want to noindex. Apparently this will only work on the latest release of Magento.
<meta name="robots" content="NOINDEX,NOFOLLOW"/>
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Pages not indexable?
Hello, I've been trying to find out why Google Search Console finds these pages non-indexable: https://www.visitflorida.com/en-us/eat-drink.html https://www.visitflorida.com/en-us/florida-beaches/beach-finder.html Moz and SEMrush both crawl the pages and show no errors but GSC comes back with, "blocked by robots.txt" but I've confirmed it is not. Anyone have any thoughts? 6AYn1TL
Technical SEO | | KenSchaefer0 -
Tracing Redirects to a Site
I wonder if anyone has used any tools where you can trace the redirects pointing to a site? I know there are a number of tools out there that can be used to check where a URL redirects to, but I was wondering if anyone has used a tool where I could trace all redirects with the final URL? I am using this for competitor research so I don't have access to Analytics or Webmaster Tools.
Technical SEO | | BeattieGroup0 -
Cache Not Working on Our Site
We redesigned our site (www.motivators.com) back in April. Ever since then, we can't view the cache. It loads as a blank, white page but the cache text is at the top saying: "This is Google's cache of http://www.motivators.com/. It is a snapshot of the page as it appeared on Jul 22, 2013 15:50:40 GMT. The current page could have changed in the meantime. Learn more. Tip: To quickly find your search term on this page, press Ctrl+F or ⌘-F (Mac) and use the find bar." Has anyone else ever seen this happen? Any ideas as to why it's happening? Could it be hurting us? Advice, tips, suggestions would be very much appreciated!
Technical SEO | | Motivators0 -
Why is this site ranking so good?
Site in question: http://bit.ly/aBvVbm Our main competitor in the UK seems to be ranking extremely good for the keyword "jigsaw puzzles" even though their linking profile doesn't seem all that great? They mainly have site-wide links on 2 of their other ecommerce sites which seem to be given them their ranking power as this equals to 100's of links. Does sitewide links on 2 sites really give this much ranking power or am I missing something?
Technical SEO | | Tonyy30 -
Site Purchase and 301
Hello, I just started working with a new client. Since then the client has purchased another company. We have re-branded the new companies home page and 301 redirected the rest of the site's links to the corresponding pages on the holding companies site. Since then the rankings have tanked. I looked at both companies back link profiles and realized that they are quite spammy from the last SEO contractor they hired. That said, the site was ranking fine until last Friday. I was wondering if anyone had seen temporary rankings decrease after 301ing a domain to a different site? Thanks!
Technical SEO | | TargetClick0 -
Getting Posts Indexed
On a Wordpress site I'm working on you can get to any product from home in 2 clicks but I'm a llittle concerned about the URL which looks like this: domain/categoryname/subcategoryname/productpage Will I have trouble getting my products indexed?
Technical SEO | | waynekolenchuk0 -
Help changing category and page titles on established Magento site.
Hi All, This is my first post, I've been a Pro member for a while now, read many posts but have never asked a question here. I have an established Magento site that I need to do a lot of SEO work on. I am a newbie at SEO, so I wanted to ask for advice here. I just changed one of the category names, the url, meta title, and meta description. When making the change I selected the checkbox in the Magento backend that says "Create Permanent Redirect for Old URL" Example:
Technical SEO | | TrulyLuna
Old Category: http://www.domain.com/old-category-name.html
New Category: http://www.domain.com/new-category-name.html Everything worked fine with the exception of one alarming thing... I lost all of the page authority for the category itself and all of the sub categories and pages below it. Now the category and all pages below it show (on the moz toolbar) a PA of 1 and 0 links and 0 domains. I updated the sitemap. I did not do a 301 redirect in the .htaccess, I only chose the option to redirect on the Magento backend when changing the name and url of the category. I need to change a lot of other category and page names and urls but now I'm a bit gun shy, as I do not want to do something that will damage what little page rank I have at the moment. I'm looking for some advice from one of you guys who might be able to help me do this the correct way. I did some research on Google, but I'm still not sure of the correct method to accomplish this. Thanks in advance for any help that you can offer.0
