Some web 2.0 sites blocked by robot
-
I had created some web2.0 sites, google also indexed those sites but those sites robots.txt are disallow, this mean Google will not give any credit on ranking ? I have seen in google, those sites meta description was written - A description of this site is not available due to the robot.txt file. I checked that file, it is written disallow. Can google still count those sites for the ranking purpose ?
-
Hi,
Pages blocked by robots.txt file will not be considered for ranking. You still see those pages in Google's index is because of their mentions on other websites. Google has been doing link discovery from websites other than these Web2.0 sites of yours. So to stop Google completely from indexing these pages, you should go in for page-levelrobots meta tag. Here you go for more: https://support.google.com/webmasters/answer/93710?hl=en
Once you implement the page-level robots meta tag to block those pages, Google will remove them from the index after the next crawl. You can expedite this process by using the link removal procedure in webmaster tools account.
Here you go for more: https://support.google.com/websearch/troubleshooter/3111061?rd=1
Good Luck to you my friend.
Best regards,
Devanur Rafi
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Free tool which includes a link back to my site
On one of my sites I provide a free tool to other webmasters to use that allows them to copy some code to use the free tool on their website and under the tool on it says "Powered by: mysite.co.uk" with a link to "mysite.co.uk". Now I have no real control over which sites can come and use this free tool (a tool to test how fast an internet connection is), on the whole though it is aimed at a specific niche but a range of sites do use it which means good or crap sites can pick up the free code to place into their sites. The link to mysite.co.uk is now always mysite.co.uk as the anchor text, I used to use the websites name (which is the actual name of the URL - link to site at end of post) so the links were keyword rich but only to my sites genuine name, however I now just leave it as the URL to my site. I am not aiming to do anything untowards with the linking other than to get the acknowledgement that it was provided for free by my site, however I'm wondering if google etc will be able to tell they are natural and not dodgy? My site is: http://bit.ly/MsNQwS (n.b. a new improved test is being created before anyone says it's not accurate). 🙂
Link Building | | Wardy0 -
How do you handle a large site with a very high DA?
I am looking at working with a site that has a DA of 93, they competitors have a DA of 90, 91, 92. Any insights to the struggles I might face with a site like this?
Link Building | | LesleyPaone0 -
Press Release Sites
I don't do that many press releases, but I need to do one soon. I had been using PRweb.com and I see they raised their prices. Are they considered the best online PR site or are there sites people have used that have gotten similar results for less money?? Thanks
Link Building | | ClickIt0 -
External links - link to third party sites
Greetings! Actually, got an doubt on linking to external site. The thing, i have working for roof marketers site. They have covered roof products. Each of the products have a page along with major keyword. Actually, i do link from other pages[internally]. Meaning,if i see any main keywords in different page then will link that to corresponding page/product[internally].This is what i am doing to get page rank. Now, my doubt. Some of the pages are having link with main keywords which directs to the third party site those who are really producing the products. But, i remove the third party-link which has with main keywords. Since already the site/my client site having pages for that main keywords. But client really want that link. meaning the links which they have given to direct to third party[product producer] site. So what should i do at this case. Can i just past RAW link like [http://www.thirdpartysite.com] or what do i do. But my client wants that link to be in content area. Hope you would understand my long explanation and case Please help us. Thank you
Link Building | | Webworld_Norway0 -
High PR Web 2.0 sites
Hello, I was wondering if you could provide some advice on which are the best Top 20 high pr web 2.0 properties for content besides wordpress? Whilst i do intend to use them to further improve my SEO campaign I also intend to use readable content, that is to say, content that i have written and i feel is worthy of being read by interested parties. What are the top 20 to look at that would benefit an SEO campaign if used correctly (without abusing or being spammy). I can see lots of crappy ones out there but they all look spammy to me and im not really liking that.
Link Building | | irdeto0 -
Following Penguin, what is the best practice for a web design agency placing a footer link in client sites?
For years we have included a 'Web Design by ...' link in the footer of all client websites designed by us. However, following the Penguin update, it seems that this legitimate practice could now be penalised. Should we be removing those links from our client's sites? There are tens of thousands, all with similar or identical text! For future client sites, should we avoid these footer links altogether? Or maybe limit the link to a single page on each site?
Link Building | | LewisJenkins691 -
Risks with sharing my original content with other sites
We write our own unique content to our blog a few times a week. The content is written by professionals (doctors and such) and carries a bit of authority. We periodically get requests from other websites who want to post our content on their blog. We also find that some people will just go ahead and use our exact content (often with a link attribution), without asking permission. Is there any risk of us not getting credit for being the original source of the content if others use it in its entirety?Are there other risks? And more importantly, what should we do to make sure we're getting the SEO credit we deserve? Some ideas which have been discussed that may or may not have credence: Only let others use the exact content well after we've been crawled, so we'll be seen as the first publisher Only let people cite the introductory paragraph, with a link allowing them to read more at our site (with good anchor text) Require people to summarize our story, with a link for the full story Any thoughts would be appreciated. Thanks!
Link Building | | OneMedical0 -
What is best way to evaluate external links to my site?
What is the best way to evaluate external links coming in to my site? I have been working on trying to build links by commenting on blogs/forums and adding to directories, etc. I've run link reports through Open Site Explorer and Yahoo and they never seem to show the same links coming in. For example, i run link eport on Open Site Explorer and it will show links coming into my site that Yahoo doesn't show, and vice versa. I guess I'm trying to figure best way to monitor each week new links coming into my site to determine if my efforts are fruitful. Also, is there a certain period of time that should pass from time I put my link on another site (ie, a blog) until it will show up in OPen Site Explorer? Thanks! Oh, and I use the NoDoFollow Add-on for Firefox to only comment on those sites that do follow. Not sure if that's really effective or not.
Link Building | | leonenobleseate0
