Some Tools Not Recognizing Meta Tags
-
I am analyzing a site which has several thousands of pages, checking the headers, meta tags, and other on page factors. I noticed that the spider tool on SEO Book (http://tools.seobook.com/general/spider-test) does not seem to recognize the meta tags for various pages. However, using other tools including Moz, it seems the meta tags are being recognized.
I wouldn't be as concerned with why a tool is not picking up the tags. But, the site suffered a large traffic loss and we're still trying to figure out what remaining issues need to be addressed. Also, many of those pages once ranked in Google and now cannot be found unless you do a site:// search.
Is it possible that there is something blocking where various tools or crawlers can easily read them, but other tools cannot. This would seem very strange to me, but the above is what I've witnessed recently.
Your suggestions and feedback are appreciated, especially as this site continues to battle Panda.
-
Hi,
I just run those with Screaming Frig - as a basic crawler - all is good. Source code also looks ok. There are no issues there - you should be safe.
I see the issues on the seobook crwler - it's for sure an issue there not on your side.
Cheers.
-
Sure. Try going to http://tools.seobook.com/general/spider-test to run a test on:
http://www.maxandchloe.com/bridal-jewelry
http://www.maxandchloe.com/nashelle
These are just two example where the meta tags nor text on the page seems to be getting picked up.
-
Hi.
Can you give some exact examples ? It will be easier to look into.
Cheers.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Meta robots
Hi, I am checking a website for SEO and I've noticed that a lot of pages from the blog have the following meta robots: meta name="robots" content="follow" Normally these pages should be indexed, since search engines will index and follow by default. In this case however, a lot of pages from this blog are not indexed. Is this because the meta robots is specified, but only contains follow? So will search engines only index and follow by default if there is no meta robots specified at all? And secondly, if I would change the meta robots, should I just add index or remove the meta robots completely from the code? Thanks for checking!
Intermediate & Advanced SEO | | Mat_C0 -
No index detected in robots meta tag GSC issue_Help Please
Hi Everyone, We just did a site migration ( URL structure change, site redesign, CMS change). During migration, dev team messed up badly on a few things including SEO. The old site had pages canonicalized and self canonicalized <> New site doesn't have anything (CMS dev error) so we are working retroactively to add canonicalization mechanism The legacy site had URL’s ending with a trailing slash “/” <> new site got redirected to Set of url’s without “/” New site action : All robots are allowed: A new sitemap is submitted to google search console So here is my problem (it been a long 24hr night for me 🙂 ) 1. Now when I look at GSC homepage URL it says that old page is self canonicalized and currently in index (old page with a trailing slash at the end of URL). 2. When I try to perform a live URL test, I get the message "No: 'noindex' detected in 'robots' meta tag" , so indexation cant be done. I have no idea where noindex is coming from. 3. Robots.txt in search console still showing old file ( no noindex there ) I tried to submit new file but old one still coming up. When I click on "See live robots.txt" I get current robots. 4. I see that old page is still canonicalized and attempting to index redirected old page might be confusing google Hope someone can help to get the new page indexed! I really need it 🙂 Please ping me if you need more clarification. Thank you ! Thank you
Intermediate & Advanced SEO | | bgvsiteadmin1 -
How Important is Meta Description on Non-Entry Pages?
If a page is never used as an entry page to your website -- in other words it's an obscure, relatively unimportant page that never ranks high enough in the search engines to be in the first few pages of the search results for any significant number of searches -- does editing the META Description really have any significant benefit? I guess the question could also be phrased as, does the Google Search Algo factor in the META Description tag, or is it only used for display purposes on the search results and doesn't affect ranking?
Intermediate & Advanced SEO | | SeoJaz0 -
Pagination, Canonical Tag & Best Practices
I have an eCommerce site that dynamically creates category pages, which produce canonical tags in the header. For multiple page categories, it adds the page number to the URL. For example, this category has 3 pages.... Because most categories have too many products, I can't follow Googles suggestion of creating a "view all" page. Furthermore since all these pages use the same template, I'm unable to insert a NOINDEX tag in all the pages after the first page. Also, in this scenario, I'm unable to insert the discreet code for Next/Previous, which is also suggested by Google. My only option for maintaining these dynamically generated category pages would be to hardcode the first conical tag in the template, which would then be produced on all subsequent paginated pages. Consequently, every paginated page in this category would have the same canonical tag pointing to the first page. Would this incur the wrath of Google and would I'd be better off leaving the pagination they way it is?
Intermediate & Advanced SEO | | alrockn0 -
Wordpress Tag Pages - NoIndex?
Hi there. I am using Yoast Wordpress Plugin. I just wonder if any test have been done around the effects of Index vs Noindex for Tag Pages? ( like when tagging a word relevant to an article ) Thanks 🙂 Martin
Intermediate & Advanced SEO | | s_EOgi_Bear0 -
Canonical Tags?
I read that Google will "honor" these tags if your website has two url's with duplicate content. The duplicate content does not show up in my SEOmoz crawls report but they do in the search engines and many of "non authoritative links" that are generated from my search feature j(ugly url's with % ...not real user friendly) are ranking higher than the "good URL" links. So if I do the canonical tags I guess my higher ranking bad urls will drop. I even read that google might even completely overlook the links. I read somewhere that the best way to do this is with a 301 redirect...is that correct? I m ranking pretty good with my main keyword terms so I am afraid to make changes not knowing the effect. Any suggestions? Thanks, Boo
Intermediate & Advanced SEO | | Boodreaux0 -
Canonical tag vs 301
What is the reason that 301 is preferred and not rel canonical tag when it comes to implementing redirect. Page rank will be lost in both cases. So, why prefer one over the other ?
Intermediate & Advanced SEO | | seoug_20050 -
What is the downside with having too long of a title tag?
With Google placing so much relevance on title tags, it seems to help if you mention local cities within your title tag. I'm wondering if the positive of having more keywords in a title tag outweighs the negative of having too long of one?
Intermediate & Advanced SEO | | TLSNET0