Please help to identify the following bots and spiders
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you help me figure out why my website is in conflict with guidelines of Google ?
Hi, Since few weeks now we received this message saying that our website is in conflict with the guidelines of Google's Webmaster. Here is the website for which we received the message from Google: http://www.gocustomized.es/. This url redirect to https://www.gocustomized.es/ I thought after reading your some messages from the forum that the problem was our website reviews which appear on all pages of the site. And I know that the review of the websites shouldn't be considered like the review of the product. But we remove the reviews and our request is still declined. Thanks in advance for your help.
Technical SEO | | steph_ba0 -
Link in my blog to external pages - no follow?
Hi everyone. I'm quite new to SEO (started 6 months ago, but I'm very lucky to work the company that is willing to pay while I'm learning). I also create some content for the website - I write wood flooring industry blog. Following some advise from few SEO experts around the world whom I follow I have decided to write 1-2 off topic blogs/month but I have found the way of writing off topic but in reality bringing together wood flooring industry and let's say fashion industry. My question is - if I want to link to one or two pages (let's say bug fashion brands) shall I use no-follow link? I do not want to harm my website or theirs. Can those type of posts be the part of building my position within my industry and in general, building authority of my website? Of course appart from getting links TO my website. Tom
Technical SEO | | ESB0 -
Rel="Follow"? What the &#@? does that mean?
I've written a guest blog post for a site. In the link back to my site they've put a rel="follow" attribute. Is that valid HTML? I've Googled it but the answers are inconclusive, to say the least.
Technical SEO | | Jeepster0 -
SEO Issues with Avactis Shopping Cart Please Help(Resolved)
I am seeking advise or help with several issues I am having with Avactis. First is Title Duplicates or adds the same on every CMS page Seal It Green Sealants - Non-Toxic Organic Sealant Products so for instance it say Contact Us-Seal It Green Sealants - Non-Toxic Organic Sealant Products or Whats the Difference Seal It or Xtreme -Seal It Green Sealants - Non-Toxic Organic Sealant Products every cms page it adds Seal It Green Sealants - Non-Toxic Organic Sealant Products to the title description. which in turn makes my Title Descriptions TOO LONG! Second is the very lengthy urls. I came into this project after the site was developed and there are more issues than I ever imagined. Duplicated Pages, Titles etc all over the place. Trying to work on one issue at a time. ( I think re-building the site using something else would of been a quicker solution) But Client didn't want to do that. Any advise or tips to get through this process with this platform would be extremely appreciated.
Technical SEO | | MACameron0 -
Help With Analytics Data
Hello, I'm seeing the following Analytics data for some of my keywords: Multiple Visits Pages/Visit: 1 Avg Visit Duration: 00:00 % New Visits: 100% Bounce Rate: 100% The data is the same on all "affected keywords". What is going on and how do I fix it? Thanks for the help!
Technical SEO | | AWCthreads0 -
Help needed with robots.txt regarding wordpress!
Here is my robots.txt from google webmaster tools. These are the pages that are being blocked and I am not sure which of these to get rid of in order to unblock blog posts from being searched. http://ensoplastics.com/theblog/?cat=743 http://ensoplastics.com/theblog/?p=240 These category pages and blog posts are blocked so do I delete the /? ...I am new to SEO and web development so I am not sure why the developer of this robots.txt file would block pages and posts in wordpress. It seems to me like that is the reason why someone has a blog so it can be searched and get more exposure for SEO purposes. IS there a reason I should block any pages contained in wodrpress? Sitemap: http://www.ensobottles.com/blog/sitemap.xml User-agent: Googlebot Disallow: /*/trackback Disallow: /*/feed Disallow: /*/comments Disallow: /? Disallow: /*? Disallow: /page/
Technical SEO | | ENSO
User-agent: * Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/themes/ Disallow: /trackback Disallow: /commentsDisallow: /feed0 -
Does Google follow a text based URL with no anchor tags?
I am seeing with Webmaster Tools that Google is trying to follow the text based truncated URL from SuperPages despite the fact that they are not in an anchor tag. The net result for Google is a 404 error as they try to access pages that do not exist. has anyone seen this issue before and any suggestions on how to prevent these errors? A Superpage listing: The first link works fine, but the text based link shown below is cut off and as a result Google gets to a 404 page. http://swbd-out.superpages.com/webresults.htm?qkw=dr+hylton+lightman&qcat=web&y=0&x=0& Dr. Hylton Lightman, MD - Pediatrician, Allergist, - Far Rockaway ... Dr. Hylton Lightman, MD, Far Rockaway, NY, Rated 4/4 By Patients. 26 Reviews, Patients' Choice Award Winner, Phone Number & Practice Locations www.vitals.com/doctors/Dr_Hylton_Lightma... [Found on Bing]
Technical SEO | | irvingw0 -
Need Help With Robots.txt on Magento eCommerce Site
Hello, I am having difficulty getting my robots.txt file to be configured properly. I am getting error emails from Google products stating they can't view our products because they are being blocked, and this past week, in my SEO dashboard, the URL's receiving search traffic dropped by almost 40%. Is there anyone that can offer assistance on a good template robots.txt file I can use for a Magento eCommerce website? The one I am currently using was found at this site here: e-commercewebdesign.co.uk/blog/magento-seo/magento-robots-txt-seo.php - However, I am getting problems from Google now because of it. I searched and found this thread here: http://www.magentocommerce.com/wiki/multi-store_set_up/multiple_website_setup_with_different_document_roots#the_root_folder_robots.txt_file - But I felt like maybe I should get some additional help on properly configuring a robots for a Magento site. Thanks in advance for any help. Please, let me know if you need more info to provide assistance.
Technical SEO | | JerDoggMckoy0