Pages being flagged in Search Console as having a "no-index" tag, do not have a meta robots tag??
-
Hi,
I am running a technical audit on a site which is causing me a few issues.
The site is small and awkwardly built using lots of JS, animations and dynamic URL extensions (bit of a nightmare).
I can see that it has only 5 pages being indexed in Google despite having over 25 pages submitted to Google via the sitemap in Search Console. The beta Search Console is telling me that there are 23 Urls marked with a 'noindex' tag, however when i go to view the page source and check the code of these pages, there are no meta robots tags at all - I have also checked the robots.txt file.
Also, both Screaming Frog and Deep Crawl tools are failing to pick up these urls so i am a bit of a loss about how to find out whats going on.
Inevitably i believe the creative agency who built the site had no idea about general website best practice, and that the dynamic url extensions may have something to do with the no-indexing.
Any advice on this would be really appreciated. Are there any other ways of no-indexing pages which the dev / creative team might have implemented by accident? - What am i missing here?
Thanks,
-
Hey,
Try fetch and render from Google Search Console. There you could check if there is an X-Robots-Tag in the response header.
For reference: https://developers.google.com/search/reference/robots_meta_tag
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Search Console - Mobile Usability Errors
A site I'm looking at for a client had 100's of pages flagged as having Mobile Usability errors in Search Console. I found that the theme uses parameters in the URLs of some of theme resources (.js/.css) to identify the version strings. These were then being blocked by a rule in the robots.txt: "Disallow: /*?" I've removed this rule, and now when I inspect URLs and test the live versions of the page they are now being reported as mobile friendly. I then submitted validation requests in Search Console for both of the errors ("Text to small" and "Clickable Elements too close") My problem now, is that the validation has completed and the pages are still being reported as having the errors. I've double checked and they're find if I inspect them individually. Does anyone else have experience clearing these issues in Search Console? Any ideas what's going on here!
Technical SEO | | DougRoberts1 -
Why are only a few of our pages being indexed
Recently rebuilt a site for an auctioneers, however it has a problem in that none of the lots and auctions are being indexed by Google on the new site, only the pages like About, FAQ, home, contact. Checking WMT shows that Google has crawled all the pages, and I've done a "Fetch as Google" on them and it loads up fine, so there's no crawling issues that is standing out. I've set the "URL Parameters" to no effect too. Also built a sitemap with all the lots in, pushed to Google which then crawled them all (massive spike in Crawl rate for a couple days), and still just indexing a handful of pages. Any clues to look into would be greatly appreciated. https://www.wilkinsons-auctioneers.co.uk/auctions/
Technical SEO | | Blue-shark0 -
Sitemap indexed pages dropping
About a month ago I noticed my pages indexed from my sitemap are dropping.There are 134 pages in my sitemap and only 11 are indexed. It used to be 117 pages and just died off quickly. I still seem to be getting consistant search traffic but I'm just not sure whats causing this. There are no warnings or manual actions required in GWT that I can find.
Technical SEO | | zenstorageunits0 -
Why blocking a subfolder dropped indexed pages with 10%?
Hy Guys, maybe you can help me to understand better: on 17.04 I had 7600 pages indexed in google (WMT showing 6113). I have included in the robots.txt file, Disallow: /account/ - which contains the registration page, wishlist, etc. and other stuff since I'm not interested to rank with registration form. on 23.04 I had 6980 pages indexed in google (WMT showing 5985). I understand that this way I'm telling google I don't want that section indexed, by way so manny pages?, Because of the faceted navigation? Cheers
Technical SEO | | catalinmoraru0 -
Magento CMS Page Meta Titles?
Hey All, I'm experiencing a bit of a problem with creating a custom, separate Meta Title for CMS pages in Magento 1.6.1.0. I know this can be added through _CMS => Page => Page Information => Page Title, _but this method presents issues, as a length of 70 characters is a little too long for breadcrumbs. I would like a way to add a second specific meta title field that overrides this first Page Title field_. _ I did find something that does this in Magento Connect for around $50.00USD, but this seems like something that should be easy and free to do. Any suggestions as to a workaround or alternative free plugin would be most welcome! Thanks!
Technical SEO | | G2W0 -
Meta title Tag dilemma.... need help
Hey, Guys I have a dilemma that I cannot figure out how to solve. One thing that I have learned is that the meta tag is probably one of the most important factors of SEO. I work in the industry of real estate and we are located in a mid-sized market, Augusta, GA, which does not have a hugely competitive digital marketplace. So, I have told my web developer the changes that I want her to make to our major sub-domain pages on our website. I am anticipating that once she makes these changes which will allow me to make the necessary SEO changes to website, that we will see some good results. I have one dilemma that I can't figure out how to solve with the meta title tag. Check out our rental section: http://aubenrealty.com/rentals.cfm Now, click on any rental property and it will take you to that rental's page. Notice the page title " Auben Realty- real estate....." This is identical for every active and non-active property on our website. Every time we create a new property, this is what it spits out. Now, take it a step further and click on " Contact me about this property," and you will see the same page title. My dilemma is, " How do we fix this?" My assumption is that the best page title would be the address for each property( ex, 1322 Laurel Street, Augusta Ga 30904), right ? Is this some kind of simple coding adjustment?
Technical SEO | | AubbiefromAubenRealty0 -
Sitemap all of a sudden only indexing 2 out of 5000+ pages
Any ideas why this happened? Our sitemap looks the same. Also, our total number of pages indexed has not decreased, just the sitemap. Could this eventually affect my pages being in the index?
Technical SEO | | rock220 -
Duplicate Title Tags and Meta Desc even with the correct Canonical Tag
I show a large/growing number of duplicate title tags and duplicate meta descriptions in my webmaster tools. I look at both pages Link 1 - http://www.thatsmytopper.com/wedding-cake-toppers/theme-cake-toppers/beach-theme-cake-toppers/where/color/petal-pink.html Link 2 - http://www.thatsmytopper.com/wedding-cake-toppers/theme-cake-toppers/beach-theme-cake-toppers/where/color/petal-pink/limit/16.html Both pages have the following canonical url: <link rel="<a class="attribute-value">canonical</a>" href="http://www.thatsmytopper.com/wedding-cake-toppers/theme-cake-toppers/beach-theme-cake-toppers.html" > Why does this show up as a duplicate title tag and description to Google still?
Technical SEO | | bhalverson0