Is Googlebot ignoring directives? Or is it Me?
-
I saw an answer to a question in this forum a few days ago, that said it was a bad idea to use robots.txt to tell googlebot to go away.
That SEO said it was much better to use the META tag to say noindex,nofollow.
So I removed the robots directive and added the META tag
<meta robots='noindex,nofollow'>
Today, I see google showing my send to a friend page where I expected the real page to be.
Does it mean Google is stupid?
Does it mean google ignores the Robots META tag?
Does it mean short pages have more value than long pages?
Does it mean if I convert my whole site to snippets, I'll get more traffic?
Does it mean garbage trumps content?
I have more questions, but this is more than enough.
-
Thank you Ryan.
They completely ignored the meta tags., completely messing up our serps. So I put it back in robots. I wont trust google again to do the right thing.
-
Hi Allan,
It is a best practice to use meta tags to indicate your indexing preference to search engines.
Normally the recommended implementation would be "noindex, follow" but without examining your site it is impossible to know for sure.
Google honors meta tags but there are a number of issues which could be the source of your issue. For example, if you did not use valid syntax the tag may not be honored. If you are blocking the page in robots.txt, then search engines cannot read the tag.
As for the last three questions, the simple answer is quality content is best.
If you can share the URL of the page involved, we can offer a specific response to the implementation of the meta tag.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz gives me an A, Goggle completely ignores
I'm sure this type of question gets asked a lot, but I need some help here. Site: www.lodgmate.com We sell hotel supplies. Site has been active over a year. It has never been listed by Google for term "hotel supplies", at all. Have run numerous analytics, including Moz. While not perfect, no major flags. Moz gives the page an "A" for "hotel supplies", but none of the search engines return this site for that term. Did have an issue with too many key words, but that has since been corrected. Have never engaged in any questionable SEO tactics. I can understand not placing high in search, but not at all?? What's wrong? Any help or suggestions would be greatly appreciated!
Technical SEO | | mhans0 -
Wordpress 4xx errors from comment re-direct
About a month ago, we had a massive jump in 4XX errors. It seems the majority are being caused by the comment tool on wordpress, which is generating a link that looks like this "http://www.turnerpr.com/blog/wp-login.php?redirect_to=http%3A%2F%2Fwww.turnerpr.com%2Fblog%2F2013%2F09%2Fturners-crew-royal-treatment-well-sort-of%2Fphoto-2-2%2F" On every single post. We're using Akismet and haven't had issues in the past....and I can't figure out the fix. I've tried turning it off and back on; I'm reluctant to completely switch commenting systems because we'd lose so much history. Anyone seen this particular re-direct love happen before? Angela
Technical SEO | | TurnerPR0 -
Are my canonical re directs working?
Buonjourno from Wetherby UK 🙂 Ive been battlling sometime to get this site http://www.goldsboroughestates.co.uk to rank for term Right To Manage. Amongst other tactics ive set up a canonical http://www.goldsboroughestates.co.uk/about/right-to-manage.aspx * - Canonical version http://www.goldsboroughestates.co.uk/how-we-care-for-you/right-to-manage.aspx http://www.goldsboroughestates.co.uk/what-our-customers-say/right-to-manage.aspx http://www.goldsboroughestates.co.uk/ - But has this canonical redirect feature worked? The reason i doubt it is i notice when i enter a page http://www.goldsboroughestates.co.uk/what-our-customers-say/right-to-manage.aspx which has the below code in place: rell="canonical" href="http://www.goldsboroughestates.co.uk/About/right-to-manage.aspx/" /> It does not jump to http://www.goldsboroughestates.co.uk/about/right-to-manage.aspx So my question is... "is the canonical redirect working or not & waht is the best way / tool for diagnosing a canonical error" Grazie tanto,
Technical SEO | | Nightwing
David0 -
WMT - Googlebot can't access your site
Hi On our new website which is just a few weeks old upon logging into Webmaster tools I am getting the following message Googlebot can't access your site - The overall error rate for DNS queries is 50% What do I need to do to resolve this, I have never had this problem before with any of the sites - where the domains are with Fasthosts (UK) and hosting is with Dreamhosts. What is the recommended course of action Google mention contacting your host in my case Dreamhost - but what do you need to ask them in a support ticket. When doing a fetch in WMT the fetch status is a success?
Technical SEO | | ocelot0 -
How to Block Urls with specific components from Googlebot
Hello, I have around 100,000 Error pages showing in Google Webmaster Tools. I want to block specific components like com_fireboard, com_seyret,com_profiler etc. Few examples: http://www.toycollector.com/videos/generatersslinks/index.php?option=com_fireboard&Itemid=824&func=view&catid=123&id=16494 http://www.toycollector.com/index.php?option=com_content&view=article&id=6932:tomica-limited-nissan-skyline-r34--nissan-skyline-gt-r-r34-vspec&catid=231&Itemid=634 I tried blocking using robots.txt. Just used this Disallow: /com_fireboard/
Technical SEO | | TheMartingale
Disallow: /com_seyret/ But its not working. Can anyone suggest me to solve this problem. Many Thanks Shradda0 -
Purchasing a site for a 301 Re-direct
Hi Mozzers, I have a question regarding a tactic I'm considering for a client. My client has a web hosting company and is ranking well for his keywords and in position 3 for his main term. There is a site available on flippa that is a keyword rich domain and has a decent link portfolio and domain authority and the price is attractive. I'm considering buying it to 301 it to his domain but I've never done this tactic before. Is this grey/black hat? Has anyone done this before and to what extent did it work? Thanks Bush
Technical SEO | | Bush_JSM0 -
Is it possibly to use anything besides a 302 re-direct when your doing a re-direct for someone to login?
Hopefully this makes sense. So I am working on a site that uses a 302 re-direct for logins. As in it goes from a profile page to the login via a re-direct, most of the time I see sites use this as a meta refresh, but in this case I wasn't sure. Obviously when I run a crawl diagnostic I'm getting a lot of errors as in over 100. Now I know there is no link juice with this, but I was just wondering what other people thought on using 302's for logins? Thanks
Technical SEO | | kateG12980 -
301 redirecting some pages directly, and the rest to a single page
I've read through the Redirect guide here already but can't get this down in my .htaccess I want to redirect some pages specifically (/contactinfo.html to the new /contact.php) And I want all other pages (not all have equivalent pages on the new site) to redirect to my new (index.php) homepage. How can I set it up so that some specific pages redirect directly, and all others go to one page? I already have the specific oldpage.html -> newpage.php redirects in place, just need to figure out the broad one for everything else.
Technical SEO | | RyanWhitney150