X-robots tag causing no index issues
-
I have an interesting problem with a site which has an x-robot tag blocking the site from being indexed, the site is in Wordpress, there are no issues with the robots.txt or at the page level, I cant find the noindex anywhere.
I removed the SEO plug-in which was there and installed Yoast but it made no difference.
this is the url: https://www.cotswoldflatroofing.com/
Its coming up with a HTTP error: x-robots tag noindex, nofollow, noarchive
-
Hi Simon,
I hope you are well.Is that an error you're getting in Search Console?
I'm checking your site, and I can't replicate that issue you are having with x-robots.
I have tried with Screaming Frog and my command line. None of them returned noindex. In the image attached, you'll see what I got.Also, I tried a site: search, and your website is indexed. You have only one page, is that correct?
Other image attachedHope it helps.
Best luck
Gastón
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Setting up hreflang tags
Hi everyone, A quick question about setting up your Hreflang tags. Here you can see 2 examples: As you can see, the order of the elements is different. Be aware, there is a tiny difference between the 2: the first Hreflang is written for a specific language in a specific country, the second one only contains a language code. Is this the reason why the order is different or is this just a coincidence and doesn't the order of the elements matter at all? Thanks,
Technical SEO | | WeAreDigital_BE
Jens0 -
I have a Category and Tag In My Blogs
I have use category and Tags in my blogs. Now i have an problem with blog URL and Tags URL. My blog URLs is also show in Tags page and both the content is same. For Example: My Blog URL is: https://www.example.com/advice-how-to-do-batting And Tag Page URL is : https://www.example.com/advice-batting in that - https://www.example.com/advice-how-to-do-batting The URLs contain same content. No should i write two different meta title and description for above two URLs pages. As there might more blog added under Tags pages with different topics and title. Request on Thought Please.
Technical SEO | | ProcessSEO0 -
Robots.txt on subdomains
Hi guys! I keep reading conflicting information on this and it's left me a little unsure. Am I right in thinking that a website with a subdomain of shop.sitetitle.com will share the same robots.txt file as the root domain?
Technical SEO | | Whittie0 -
Rankings Issues
website: trophycentral.com Back in February of this year, we took a huge rankings hit on Google. We thought it might be Panda related, but realized it was before the big change. We also thought it could be related to our Mobile rollout and a wholesale system we put in place, but were not sure. Most pages were fine, but our home page was hit hard and taken off line, included for branded terms (although the branded terms came back in a couple of weeks). There were no manual actions against us. We made a ton of changes with the mobile site, crawl issues, reducing links, reviewing and removing potentially bad outside links, etc. After a few months (long time!) almost all of our keywords came back except one, trophies. We used to rank in the top few positions and now are not in the top 500. We believe we are being penalized, but have no idea why this one keyword is being impacted. It is obviously a huge one for us and we want to get it back. Does anyone have any ideas as to why Trophies is not ranking on our home page. We are grateful that we have many other words and phrases in the top 5 positions (not pages) of google, yahoo and bing, but on Google, we cant seem to get trophies to even show up! Please help!!! Thanks!!!
Technical SEO | | trophycentraltrophiesandawards0 -
Duplicate content issue index.html vs non index.html
Hi I have an issue. In my client's profile, I found that the "index.html" are mostly authoritative than non "index.html", and I found that www. version is more authoritative than non www. The problem is that I find the opposite situation where non "index.html" are more authoritative than "index.html" or non www more authoritative than www. My logic would tell me to still redirect the non"index.html" to "index.html". Am I right? and in the case I find the opposite happening, does it matter if I still redirect the non"index.html" to "index.html"? The same question for www vs non www versions? Thank you
Technical SEO | | Ideas-Money-Art0 -
Robots.txt
Hello Everyone, The problem I'm having is not knowing where to have the robots.txt file on our server. We have our main domain (company.com) with a robots.txt file in the root of the site, but we also have our blog (company.com/blog) where were trying to disallow certain directories from being crawled for SEO purposes... Would having the blog in the sub-directory still need its own robots.txt? or can I reference the directories i don't want crawled within the blog using the root robots.txt file? Thanks for your insight on this matter.
Technical SEO | | BailHotline0 -
Robots txt
We have a development site that we want google and other bots to stay out of but we want roger to have access. Currently our robots.txt looks like this: User-agent: *
Technical SEO | | LadyApollo
Disallow: /cgi-bin/
Disallow: /development/ What would i need to addd or change to let him through? Thank you.0 -
De-indexing thin content & Panda--any advantage to immediate de-indexing?
We added the nonidex, follow tag to our site about a week ago on several hundred URLs, and they are still in Google's index. I know de-indexing takes time, but I am wondering if having those URLs in the index will continue to "pandalize" the site. Would it be better to use the URL removal request? Or, should we just wait for the noindex tags to remove the URLs from the index?
Technical SEO | | nicole.healthline0