X-robots tag causing no index issues
-
I have an interesting problem with a site which has an x-robot tag blocking the site from being indexed, the site is in Wordpress, there are no issues with the robots.txt or at the page level, I cant find the noindex anywhere.
I removed the SEO plug-in which was there and installed Yoast but it made no difference.
this is the url: https://www.cotswoldflatroofing.com/
Its coming up with a HTTP error: x-robots tag noindex, nofollow, noarchive
-
Hi Simon,
I hope you are well.Is that an error you're getting in Search Console?
I'm checking your site, and I can't replicate that issue you are having with x-robots.
I have tried with Screaming Frog and my command line. None of them returned noindex. In the image attached, you'll see what I got.Also, I tried a site: search, and your website is indexed. You have only one page, is that correct?
Other image attached
Hope it helps.
Best luck
Gastón
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Some Issues about my Blog
I am facing issue regarding to my Blog https://digitalmedialine.com/blog/. As some pages are not Rank in google yet. Can Anyone help me out how to rank those blogs to improve my Traffic. Thanks in Advance.
Technical SEO | | qwaswd0 -
Canonical tag not working
I have a weebly site and I put the canonical tag in the header code but the moz crawler still says that I'm missing the canonical tag. Any tips?
Technical SEO | | ctpolarbears0 -
Easy Question: regarding no index meta tag vs robot.txt
This seems like a dumb question, but I'm not sure what the answer is. I have an ecommerce client who has a couple of subdirectories "gallery" and "blog". Neither directory gets a lot of traffic or really turns into much conversions, so I want to remove the pages so they don't drain my page rank from more important pages. Does this sound like a good idea? I was thinking of either disallowing the folders via robot.txt file or add a "no index" tag or 301redirect or delete them. Can you help me determine which is best. **DEINDEX: **As I understand it, the no index meta tag is going to allow the robots to still crawl the pages, but they won't be indexed. The supposed good news is that it still allows link juice to be passed through. This seems like a bad thing to me because I don't want to waste my link juice passing to these pages. The idea is to keep my page rank from being dilluted on these pages. Kind of similar question, if page rank is finite, does google still treat these pages as part of the site even if it's not indexing them? If I do deindex these pages, I think there are quite a few internal links to these pages. Even those these pages are deindexed, they still exist, so it's not as if the site would return a 404 right? ROBOTS.TXT As I understand it, this will keep the robots from crawling the page, so it won't be indexed and the link juice won't pass. I don't want to waste page rank which links to these pages, so is this a bad option? **301 redirect: **What if I just 301 redirect all these pages back to the homepage? Is this an easy answer? Part of the problem with this solution is that I'm not sure if it's permanent, but even more importantly is that currently 80% of the site is made up of blog and gallery pages and I think it would be strange to have the vast majority of the site 301 redirecting to the home page. What do you think? DELETE PAGES: Maybe I could just delete all the pages. This will keep the pages from taking link juice and will deindex, but I think there's quite a few internal links to these pages. How would you find all the internal links that point to these pages. There's hundreds of them.
Technical SEO | | Santaur0 -
Canonical tag or 301
Hi, Our crawl report is showing duplicate content. some of the report I am clear about what to do but on others I am not. Some of the duplicate content arises with a 'theme=default' on the end of the URL. Is this version of a page necessary for people to see when they visit the site (like a theme=print page is) in which case I think we should use a canonical tag, or is it not necessary in which case we should use a 301? Thanks
Technical SEO | | Houses0 -
Single URL not indexed
Hi everyone! Some days ago, I noticed that one of our URLs (http://www.access.de/karriereplanung/webinare) is no longer in the Google index. We never had any form of penalty, link warning etc. Our traffic by Google is constantly growing every month. This single page does not have an external link pointing to it - only internal links. The page has been indexed all the time. The HTTP status code is 200, there is no noindex or something in the code. I submitted the URL on GWMT to let Google send it to the index. It was crawled successfully by Google, sent to the index 5 days ago - nothing happened, still not indexed. Do you have any suggestions why this page is no longer indexed? It is well linked internally and one click away from the home page. There is still the PR of 5 showing, I always thought that pages with PR are indexed.......
Technical SEO | | accessKellyOCG0 -
• symbol in title tag
We have a few title tags with a circular dot symbol, which is created by the code "•" Humans see a dot, but googlebot sees • Does this negatively impact our SEO, or is googlebot aware that **• == *** to human eyes
Technical SEO | | lighttable0 -
Backlinks Indexing
Is there a way of indexing my backlinks?? I have a lot backlinks but Google can't find them
Technical SEO | | CodePlus0 -
Canonical Issues with Wordpress
Hi all, I have just started using Wordpress SEO by Yoast and still having a hard time correcting my Canonical issues for all posts with a .html at the end. The pluggin allows you to add a '/' to the end for canonical issues, but just for pages, not posts. How best in Wordpress to make my post change from .html/ to .html. I really don't want to go to the hassle to make each URL a new 301 redirect in my .htaccess. I hate the .html, but if they are going to stay, how can I make sure I get the .html/ link juice back to them. Many thanks!
Technical SEO | | RunningInTheRain0