Not using a robot command meta tag
-
Hi SEOmoz peeps.
Was doing some research on robot commands and found a couple major sites that are not using them.
If you check out the code for these:
http://www.zappos.com/product/7787787/color/92100
You fill not find a meta robot command line.
Of course you need the line for any noindex, nofollow, noarchive pages.
However for pages you want crawled and indexed, is there any benefit for not having the line at all?
Thanks!
-
Barry is spot on. You either have the meta robot line, or you do your robots.txt. I prefer to use the txt because it shows me all the rules I impose on the website at a glance and doesn't make me go and search each page individually if anything needs to be changed. Imagine having a website with 20000 pages where each is individually adjusted. If you suddenly need to change something it will take you forever. With a robots.txt it will literally take seconds.
-
Doesn't matter if the line's not there if you don't want the robots to do anything. Not including it just assumes the position that they can do whatever they're doing ie Index, Follow
All of those sites do have a few things in robots.txt though (and the meta is really for people that can't edit robots.txt)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Multiple H2 tags
Is it advisable to use only one H2 tag? The template designs for some reason is ended up with multiple H2 tags, I realise if any think it's that each one is that are important and it is all relative. Just trying to assess if it's worth the time and effort to rehash the template. Has anyone done any testing or got any experience? Thanks
Intermediate & Advanced SEO | | seoman101 -
Robots.txt question
I notice something weird in Google robots. txt tester I have this line Disallow: display= in my robots.text but whatever URL I give to test it says blocked and shows this line in robots.text for example this line is to block pages like http://www.abc.com/lamps/floorlamps?display=table but if I test http://www.abc.com/lamps/floorlamps or any page it shows as blocked due to Disallow: display= am I doing something wrong or Google is just acting strange? I don't think pages with no display= are blocked in real.
Intermediate & Advanced SEO | | rbai0 -
Google not taking Meta...
Hello all, So I understand that Google may sometimes take content from the page as a snippet to display on SERPs rather than the meta description, but my problem goes a little beyond that. I have a section on my site which updates everyday so a lot of the content is dynamics (products for a shop, every morning unique stock is added or removed), and despite having a meta description, title and receiving an 'A' grade in the MOZ on page grader, these pages never show up in Google. After a little research I did a 'site:www.mysite.com/productpage' in Google and this indeed listed all my products, but interestingly for every single one Google had taken the copyright notice at the bottom of the page as the snippet instead of the meta or any H1, H2 or P text on the page... Does anyone have any idea why Google is doing this? It would explain a lot to me in terms of overall traffic, I'm just out of ideas... Thanks!
Intermediate & Advanced SEO | | HB170 -
Https://www.mywebsite.com/blog/tag/wolf/ setting tag pages as blog corner stone article?
We do not have enough content rich page to target all of our keywords. Because of that My SEO guy wants to set some corner stone blog articles in order to rank them for certain key words on Google. He is asking me to use the following rule in our article writing(We have blog on our website):
Intermediate & Advanced SEO | | AlirezaHamidian
For example in our articles when we use keyword "wolf", link them to the blog page:
https://www.mywebsite.com/blog/tag/wolf/
It seems like a good idea because in the tag page there are lots of material with the Keyword "wolf" . But the problem is when I search for keyword "wolf" for example on the Google, some other blog pages are ranked higher than this tag page. But he tells me in long run it is a better strategy. Any idea on this?0 -
Ecommerce question - Should I use a CDN for my images. ?
Hi , We are currently in the process of re-developing out commerce website and I wondering should we use a CDN (content delivery nertwork) for our product images. My category pages are currently showing approx 21 product images per page and the page speed is okay but can be better but the page size is rather large ... anything between 600kb - 1 Meg. We do optimise the images already in photoshop. We also do things like minify etc to get the pages to load as fast as possible but I think the only thing left is using a CDN but I have heard mixed reports about using this.? We are also doing a mobile responsive version of the site to but I know that speed will be king with google and how it reflects on rankings. Whilst I can see a CDN will improve image page load speed etc, I guess there a negative SEO impact as well as images will be stored in the cloud ?.. as opposed on to on my site/database. Does anyone know how best to implement a CDN without impacting on SEO or know of any good SEO /implementation articles on this ?... Maybe do Ieave some images on my category pages so I can still do the alt image tags etc/ and have the remaining images on the CDN.? Many Thanks Sarah
Intermediate & Advanced SEO | | SarahCollins0 -
Adding Meta Languange tag to xhtml site - coding help needed
I've had my site dinged by Google and feel it's likely several quality issues and I'm hunting down these issues. One of Bing's Webmaster SEO tools said my xhtml pages (which were built in 2007) are missing Meta Language and suggested adding tag in the  or on the html tag. Wanting to "not mess anything up" and validate correctly, I read in **W3C's site and it said: ** "Always add a lang attribute to the html tag to set the default language of your page. If this is XHTML 1.x you should also use the xml:lang attribute (with the same value). Do not use the meta element with http-equiv set to Content-Language." My current html leads like: QUESTION:
Intermediate & Advanced SEO | | mlm12
I'm confused on how to add the Meta Language to my website given my current coding as I"m not a coder. Can you suggest if I should add this content-language info, and if so, what is the best way to do so, considering valid w3c markup for my document type? Thank you!!!
Michelle0 -
Hierachy in a Title Tag Needed?
When coding the title tags of a website, is it important to note the hierarchy of the website so the search engine can find that page? So, for example, the title tag would be, for a subcategory: Webpage Subcategory>Category>Website name Does this help the search engine rankings at all? Or can the search engines figure out the hierarchy by reading a sitemap or the HTML readable navigation? Then you could focus on the descriptive keywords of the subcategory page for the title tag. Also, should you always include the site name in the title tag?
Intermediate & Advanced SEO | | ConnellyPartners0 -
Do any of you regularly use expired domains?
I know there has been discussion on using expired domains in the past. Â This is not so much a question as to how to do it or whether it works, but rather I would love to see how many of you use this in your backlink strategy. I have a domain in a low to moderately competitive niche that ranks really well, mostly on the power of a couple of expired domains. Â I bought the domains, created a quick wordpress site and pointed some anchor texted links to the site. Â It took some time for the expired domains to regain their PR, but when they did, the benefit was great. I'm considering whether I want to do this with another domain of mine. Â On one hand, it's a relatively inexpensive way to get some good quality anchor texted links. Â But, on the other hand, something in it feels "immoral" or "sneaky" to me. What do you think?
Intermediate & Advanced SEO | | MarieHaynes0