How to effectively de-index in Magento site?
-
We have thousands of Missing Description issues but most of them are account/login pages.
i.s. /customer/account/ etc...
We tried to de-index them through the Configuration using the instructions here - https://docs.magento.com/user-guide/marketing/search-engine-robots.html
But they're still appearing as issues in the Site Crawl.
Even without the site crawl issue, we don't really want these to appear in the SERPs.
Does anybody know how to properly de-index these login pages in Magento?
Thank you!
-
Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. Additionally, if you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws. -
To effectively de-index a Magento site, you can follow these steps:
Use the "Robots Meta Tag" to prevent indexing: You can add a meta tag to the header of your web pages to instruct search engines not to index them.
Use the "Robots.txt" file: You can use the robots.txt file to disallow search engine crawlers from accessing certain pages on your site.
Use the "Noindex" directive: Within the HTML code of your web pages, you can use the "noindex" directive to prevent search engines from indexing specific pages.
Use the "Canonical URL" tag: You can use the canonical URL tag to specify the preferred version of a web page, which can help prevent duplicate content from being indexed.
It's important to note that de-indexing pages should be done carefully, as it can impact your site's visibility in search engine results. If you have specific pages or sections in mind that you'd like to de-index, please let me know so I can provide more detailed guidance.
-
@LASClients Hey LASClients,
I feel your pain with those pesky login pages showing up in the Site Crawl. Have you considered using the Disallow directive in the robots.txt file to prevent search engines from crawling these pages? It's a quick fix, but as always, test it out in a staging environment first. Cheers!
Best,
[omaze cornwall] -
Certainly! To effectively de-index login pages in Magento and address the Missing Description issues, follow these steps:
Robots Meta Tag:
Open the respective login page templates, such as /customer/account/, in your Magento admin.
Add the following meta tag to the <head> section of the HTML:
html
<meta name="robots" content="noindex, nofollow">
This tag instructs search engines not to index the page and not to follow any links on it.
Robots.txt File:Edit your robots.txt file in the root of your Magento installation.
Add the following lines to disallow crawling of login pages: User-agent: *
Disallow: /customer/account/
Replace /customer/account/ with the relevant path for your login pages.
XML Sitemap:If you have an XML sitemap, ensure that the login pages are excluded from it.
Open your XML sitemap file and remove or comment out the entries related to login pages.
Submit Updated Sitemap to Search Engines:After making these changes, resubmit your updated XML sitemap to search engines via Google Search Console or Bing Webmaster Tools.
Clear Cache and Reindex:Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. If you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws. -
@get1200 @get1200
Certainly! To effectively de-index login pages in Magento and address the Missing Description issues, follow these steps:Robots Meta Tag:
Open the respective login page templates, such as /customer/account/, in your Magento admin.
Add the following meta tag to the <head> section of the HTML:
<meta name="robots" content="noindex, nofollow">
This tag instructs search engines not to index the page and not to follow any links on it.
Robots.txt File:Edit your robots.txt file in the root of your Magento installation.
Add the following lines to disallow crawling of login pages:
User-agent: *
Disallow: /customer/account/
Replace /customer/account/ with the relevant path for your login pages.
XML Sitemap:If you have an XML sitemap, ensure that the login pages are excluded from it.
Open your XML sitemap file and remove or comment out the entries related to login pages.
Submit Updated Sitemap to Search Engines:After making these changes, resubmit your updated XML sitemap to search engines via Google Search Console or Bing Webmaster Tools.
Clear Cache and Reindex:Clear your Magento cache and reindex the website to ensure that the changes take effect.
By implementing these steps, you should effectively de-index the login pages in Magento. Keep in mind that changes may take some time to reflect in search engine results. If you encounter any challenges or need further assistance, consider consulting Magento support or your web development team. Additionally, if you're interested in other Magento-related topics, you may find valuable information on Omaze Cornwall, a platform offering dream homes through exciting draws. -
@LASClients Create a file app/design/frontend/[Vendor]/[theme]/Magento_Customer/layout/customer_account_login.xml with the following content:
<?xml version="1.0"?> <page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd"> <head> <meta name="robots" content="noindex,nofollow" /> </head> </page>Clear cache
php bin/magento cache:flushAnd it should be fine.
-
@LASClients you could try adding the below meta in the pages that you want to noindex. Apparently this will only work on the latest release of Magento.
<meta name="robots" content="NOINDEX,NOFOLLOW"/>
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can adding thousands of new indexable URLs to my site at once be a problem?
Hi everyone, I am currently working on a project that will quickly add thousands of new indexable URLs to my site. For context, the site currently has over a million indexable pages. Is there any danger of adding a few thousand URLs at once to the site? Could it potentially affect crawlability/SEO/other pages? Thank you!
Technical SEO | | StevenLevine0 -
White listing a site
A new clients site is blocked by a lot of Firewalls. And I can't work out why, the content is family friendly they sell nursery equipment. I've run it through the Google checker and there is no malicious software found on the site. Can anyone tell me what I need to do to get this site unblocked? The url is http://knuma.co.uk/
Technical SEO | | Marketing_Optimist0 -
Site Link Issues
For several search terms I get site links for the page http://www.waikoloavacationrentals.com/kolea-rentals/kolea-condos/ It makes sense that that page be a site link as it is one of my most used pages, but the problem is google gave it the site link "Kolea 10A". I am having 0 luck making any sense of why that was chosen. It should be something like "Kolea Condos" or something of that nature. Does anyone have any thoughts on where google is coming up with this?
Technical SEO | | RobDalton0 -
Site Navigation
Hello, I have some questions about best practices with site navigation & internal linking. I'm currently assisting aplossoftware.com with its navigation. The site has about 200 pages total. They currently have a very sparse header with a lot of links in the footer. The three most important keywords they want to rank for are nonprofit accounting software, church accounting software and file 990 online. 1. What are your thoughts about including a drop down menu in the header for the different products? (they have 3 main products). This would allow us to include a few more links in the header and give more real estate to include full keywords in anchor text. 2. They have a good blog with content that gets regularly updated. Currently it's linked in the footer and gets a tiny amount of visits. What are your thoughts about including it as a link in the header instead? 3. What are best practices with using (or not using) no follow with site navigation and footer links? How about with links to social media pages like Facebook/Twitter? Any other thoughts/ideas about the site navigation for this site (www.aplossoftware.com) would be much appreciated. Thanks!
Technical SEO | | stageagent0 -
Canonical needed after no index
Hi do you need to point canonical from a subpage to main page if you have already marked a no index on the subpage, like when google is not indexing it so do we need canonicals now as is it passing any juice?
Technical SEO | | razasaeed0 -
Best way to handle indexed pages you don't want indexed
We've had a lot of pages indexed by google which we didn't want indexed. They relate to a ajax category filter module that works ok for front end customers but under the bonnet google has been following all of the links. I've put a rule in the robots.txt file to stop google from following any dynamic pages (with a ?) and also any ajax pages but the pages are still indexed on google. At the moment there is over 5000 pages which have been indexed which I don't want on there and I'm worried is causing issues with my rankings. Would a redirect rule work or could someone offer any advice? https://www.google.co.uk/search?q=site:outdoormegastore.co.uk+inurl:default&num=100&hl=en&safe=off&prmd=imvnsl&filter=0&biw=1600&bih=809#hl=en&safe=off&sclient=psy-ab&q=site:outdoormegastore.co.uk+inurl%3Aajax&oq=site:outdoormegastore.co.uk+inurl%3Aajax&gs_l=serp.3...194108.194626.0.194891.4.4.0.0.0.0.100.305.3j1.4.0.les%3B..0.0...1c.1.SDhuslImrLY&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.&fp=ff301ef4d48490c5&biw=1920&bih=860
Technical SEO | | gavinhoman0 -
Google not visiting my site
Hi my site www.in2town.co.uk which is a lifestyle magazine has gone under a major refit. I am still working on it but it should be ready by the end of this week or sooner but one problem i have is, google is not visiting the site. I took a huge gamble to redo the site, even though before the refit i was getting a few thousand visitors a day, i wanted to make the site better as i was getting google webmaster errors. But now it seems google is not visiting the site. for example i am using sh404sef and i have put friendly url in the site and on the home page it has its name and meta tag but when you look at google it is not giving the site a name. Also it has not visited the site since october 13th Can anyone advise how to encourage google to visit the site please.
Technical SEO | | ClaireH-1848860
