How to effectively de-index in Magento site?

LASClients

We have thousands of Missing Description issues but most of them are account/login pages.

i.s. /customer/account/ etc...

We tried to de-index them through the Configuration using the instructions here - https://docs.magento.com/user-guide/marketing/search-engine-robots.html

But they're still appearing as issues in the Site Crawl.

Even without the site crawl issue, we don't really want these to appear in the SERPs.

Does anybody know how to properly de-index these login pages in Magento?

Thank you!

lindapeter565

Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. Additionally, if you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws.

lsdmts

To effectively de-index a Magento site, you can follow these steps:

Use the "Robots Meta Tag" to prevent indexing: You can add a meta tag to the header of your web pages to instruct search engines not to index them.

Use the "Robots.txt" file: You can use the robots.txt file to disallow search engine crawlers from accessing certain pages on your site.

Use the "Noindex" directive: Within the HTML code of your web pages, you can use the "noindex" directive to prevent search engines from indexing specific pages.

Use the "Canonical URL" tag: You can use the canonical URL tag to specify the preferred version of a web page, which can help prevent duplicate content from being indexed.

It's important to note that de-indexing pages should be done carefully, as it can impact your site's visibility in search engine results. If you have specific pages or sections in mind that you'd like to de-index, please let me know so I can provide more detailed guidance.

lindapeter565

@LASClients Hey LASClients,

I feel your pain with those pesky login pages showing up in the Site Crawl. Have you considered using the Disallow directive in the robots.txt file to prevent search engines from crawling these pages? It's a quick fix, but as always, test it out in a staging environment first. Cheers!

Best,
[omaze cornwall]

lindapeter565

Certainly! To effectively de-index login pages in Magento and address the Missing Description issues, follow these steps:

Robots Meta Tag:

Open the respective login page templates, such as /customer/account/, in your Magento admin.
Add the following meta tag to the <head> section of the HTML:
html
<meta name="robots" content="noindex, nofollow">
This tag instructs search engines not to index the page and not to follow any links on it.
Robots.txt File:

Edit your robots.txt file in the root of your Magento installation.
Add the following lines to disallow crawling of login pages: User-agent: *
Disallow: /customer/account/
Replace /customer/account/ with the relevant path for your login pages.
XML Sitemap:

If you have an XML sitemap, ensure that the login pages are excluded from it.
Open your XML sitemap file and remove or comment out the entries related to login pages.
Submit Updated Sitemap to Search Engines:

After making these changes, resubmit your updated XML sitemap to search engines via Google Search Console or Bing Webmaster Tools.
Clear Cache and Reindex:

Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. If you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws.

lindapeter565

@get1200 @get1200
Certainly! To effectively de-index login pages in Magento and address the Missing Description issues, follow these steps:

Robots Meta Tag:

Open the respective login page templates, such as /customer/account/, in your Magento admin.
Add the following meta tag to the <head> section of the HTML:
<meta name="robots" content="noindex, nofollow">
This tag instructs search engines not to index the page and not to follow any links on it.
Robots.txt File:

Edit your robots.txt file in the root of your Magento installation.
Add the following lines to disallow crawling of login pages:
User-agent: *
Disallow: /customer/account/
Replace /customer/account/ with the relevant path for your login pages.
XML Sitemap:

If you have an XML sitemap, ensure that the login pages are excluded from it.
Open your XML sitemap file and remove or comment out the entries related to login pages.
Submit Updated Sitemap to Search Engines:

After making these changes, resubmit your updated XML sitemap to search engines via Google Search Console or Bing Webmaster Tools.
Clear Cache and Reindex:

Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. Additionally, if you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws.

get1200

@LASClients Create a file app/design/frontend/[Vendor]/[theme]/Magento_Customer/layout/customer_account_login.xml with the following content:

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <meta name="robots" content="noindex,nofollow" />
    </head>
</page>

Clear cache

php bin/magento cache:flush

And it should be fine.

White Rabbit

@LASClients you could try adding the below meta in the pages that you want to noindex. Apparently this will only work on the latest release of Magento.

<meta name="robots" content="NOINDEX,NOFOLLOW"/>

Source: https://community.magento.com/t5/Magento-2-x-Programming/Control-Meta-data-individually-for-Pages/m-p/404671/highlight/true#M9726

Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

How to effectively de-index in Magento site?

Got a burning SEO question?

Browse Questions

Explore more categories

Related Questions

Google Indexed a version of my site w/ MX record subdomain

How to de-index a page with a search string with the structure domain.com/?"spam"

Site indexed by Google, but (almost) never gets impressions

Indexing a catalogue

My beta site (beta.website.com) has been inadvertently indexed. Its cached pages are taking traffic away from our real website (website.com). Should I just "NO INDEX" the entire beta site and if so, what's the best way to do this? Please advise.

Why hasn't my sites indexed on opensiteexplorer.org changed in weeks?

Optimize flash site

Site just will not be reincluded in Google's Index