Is there a tool or other way to see which of my website pages employ noindex tag?
-
Hi guys
I am checking my website for possible technical issues and was wondering if there is a tool or other way to see which of my pages employ the head noindex tag if any. Do you happen to know?
Thanks
Lily
-
-
Hi Lily,
You can find this information in your moz crawl reports. If you export the report and look for the column 'Blocked by meta-robots' this should give you a list of the pages that have this tag (they will be flagged true in this column). You can also use the screaming frog tool which should give you this information in the meta and canonical tab.
-
Then you might be looking for something like this,
-
Hi
Thanks for adding up. My question is more about a tool which will list all pages which employ the noindex, not to check them one by one individually.
Thanks
-
Hello Lily,
One option is to View source of the Page and find the no-index tag there, if there is any then remove them.. or if you have fairly large site to check then its better If you use PHP, it should be fairly easy to write a script that goes through the sites, fetches the page and parses them for 'nofollow'.
Regards
Sherry
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I nofollow/noindex the outgoing links in a news aggregator website?
We have a news aggregator site that has 2 types of pages: First Type:
Technical SEO | | undaranfahujakia
Category pages like economic, sports or political news and we intend to do SEO on these category pages to get organic traffic. These pages have pagination and show the latest and most viewed news on the corresponding category. Second Type:
News headlines from other sites are displayed on the category pages. The user will be directed to that news page on the main site by clicking on a link. These links are outgoing links and we redirect them by JavaScript (not 301).
In fact these are our websites articles that just have titles (linked to destination) and meta descriptions (reads from news RSS). Question:
Should we have to nofollow/noindex the second type of links? In fact, since the crawl budget of websites is limited, isn't it better to spend this budget on the pages we have invested in (first type)?0 -
What is the best tool for getting a SiteMap url for a website with over 4k pages?
I have just migrated my website from HUGO to Wordpress and I want to submit the sitemap to Google Search Console (because I haven't done so in a couple years). It looks like there are many tools for getting a sitemap file built. But I think they probably vary in quality. Especially considering the size of my site.
Technical SEO | | DanKellyCockroach2 -
Schema markup in tag manager for multiple locations not registering in tester tool or search console
Hi Moz community, I had implemented schema markup for companies with multiple branches. We setup an organization with multiple points of contact tag, a Local Business tag for each branch and tags for specific products/ services - all through Google tag manager. I managed to fix the product markup with a small update to the code I found in a Google forum but have been unable to revive our local business markup. The tags of the schema mark up are active but when I run the Google structured data testing tool it doesn't find any schema tags. We are seeing some of the tags show up in search console but not all of them. Has anyone else had this problem and found a solution? Or, do you have any recommendations on how to markup an organization with multiple branches? Should we have one overall organization tag and a separate one for each business or is there another way of presenting each branch? Appreciate any insight!
Technical SEO | | Alexanders0 -
Purchased domain with links - redirect page by page or entire domain?
Hi, I purchased an old domain with a lot of links that I'm redirecting to my site. I want all of their links to redirect to the same page on my site so I can approach this two different ways: Entire site
Technical SEO | | ninel_P
1.) RedirectMatch 301 ^(.*)$ http://www.xyz.com or Page by page
2). Redirect 301 /retiredpage.html http://www.xyz.com/newpage.html Is there a better option I should go with in regards to SEO effectiveness? Thanks in advance!0 -
Best way to change URL for already ranking pages
Hello. I have a lot of pages that I'm optimising. The ones I'm focusing on right now is already ranking, but the URLs could be better (they don't include the keywords right now). However I'm worried that if I change the URLs they will drop in rankings or have to start over. I would of course set up 301 redirect, but is there more I need to do? What is the best way to change URL for already ranking pages?
Technical SEO | | GoMentor0 -
How to rank 1 page for multiple keywords in the new way
Hi There It has been a little while since I was involved with KW's in earnest. 1.5 years ago and beyond I did really well with SEO. I'm not in a hugely competitive market but we found our keywords, we wrote great web pages for 1,2,3 keywords and when we found more great keywords that we built a new page to rank for. For example: One big hitting keyword was "Rugged PDA", we created a category page for Rugged PDA's. Another was "Rugged Handheld" so we had a new page for that. We then long tailed "semi rugged PDA", "waterproof rugged PDA" etc etc and built sub category pages. We were legit, did lots of content marketing, ran a blog tweeted etc and we did really well to be honest. However these days it's not working, One of Rand's whiteboard sessions stated that you need to build bigger topic based pages that delivered on more keywords (The one about shoes!). This is great as we love that idea as we can have 1 big category page that offers great value to the visitor, however I am struggling to work out how we target a bigger list of keywords to the one page or to fewer pages. To underline this the MOZ page rankers also still seem to work in the same way where they expect 1 or 2 KW's per page to get A ranks to them, so I'm confused!! For example Rugged PDA is an old term, Google trends is showing that it's glory days are over and we know that the term "Rugged Smartphone" is the one to use as we all use smartphones not PDAs these days. However we also see a lot about Rugged + Phone, Mobile, Cell, Handheld, tablet, device, phablet... all relevant to one big category page. So I run these KW's through google search to see if the same pages come up as a test to see if Google thinks they all mean the same, I get a few, but not much overlap. How do we therefore have 1 page that talks about all kinds of great stuff about the "Rugged smartphone" but one that also targets rugged handheld, rugged android device etc etc? I've spent 2 days catching up, i'm none the wiser on this specific element but i'm sure I am just missing one key element of common sense here and any help is very much appreciated. Regards Dave
Technical SEO | | Raptor-crew0 -
What I doing wrong when trying to search for links from external websites to my website
This is just the little frustrating question nothing important but I’m sure somebody will know the answer. In the white board Friday this week Rand suggested at one point that when you’re searching for results links to your website if you put a - followed by site followed by your url like –site:yourwebsite.com you get the results of pages with links on other websites but excluding your own webpages but it just doesn’t work I get no results just an error message, any idea why? If I remove the - I get tons of results but there on my own webpages……….
Technical SEO | | whitbycottages0 -
Sitemap for dynamic website with over 10,000 pages
If I have a website with thousands of products, is it a good idea to create a sitemap for this website for the search engines where you show maybe 250 products on a page so it makes it easy for the search engine to find the part and also puts that part closer to the home page? Seems like google likes pages that are the closest to the home page (less clicks the better)
Technical SEO | | roundbrix0