Please help to identify the following bots and spiders
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you help by advising how to stop a URL from referring to another URL on my website with a 404 errorplease?
How to stop a URL from referring to another URL on my site. I'm getting a 404 error on a referred URL which is (https://webwritinglab.com/know-exactly-what-your-ideal-clients-want-in-8-easy-steps/[null id=43484])referred from URL (https://webwritinglab.com/know-exactly-what-your-ideal-clients-want-in-8-easy-steps/) The referred URL is the URL page that I want and I do not need it redirecting to the other URL as that's presenting a 404 error. I have tried saving the permalink in WordPress and recreated the .htaccess file and the problem is still there. Can you advise how to fix this please? Is it a case of removing the redirect? Is this advisable and how do I do that please? Thanks
Technical SEO | | Nichole.wynter20200 -
"Noindex, follow" for thin pages?
Hey there Mozzers, I have a question regarding Thin pages. Unfortunately, we have Thin pages, almost empty to be honest. I have the idea to ask the dev team to do "noindex, follow" on these pages. What do you think? Has someone faced this situation before? Will appreciate your input!
Technical SEO | | Europarl_SEO_Team0 -
Follow no-index
I have a question about the right way to not index pages: With a canonical or follow no-index. First we have a blog page: **Blogpage **
Technical SEO | | Happy-SEO
URL: /blog/
index follow Page 2 blog:
URL: /blog?=p2
index follow
rel="prev" /blog/
el="next" ?=p3 Nothing strange here i guess. But we also have other pages with chance on duplicate content: /SEO-category/
/SEO-category/view-more/ Because i don't want the "view-more" items to be indexed i want to set it on: follow no-index (follow to reach pages). But now the "view-more" also have pagination. What is the best way? Option 1:
/SEO-category/view-more/
Follow no-index /SEO-category/view-more?=p2
Follow no-index
rel="prev" /view-more/
el="next" ?=p3 Option 2: /SEO-category/view-more/
Canonical: /SEO-category/ /SEO-category/view-more?=p2
rel="prev" /view-more/
el="next" ?=p3 Option 3: Other suggests? Thanks!0 -
Steady, but continous Google traffic drop. Help?
I am facing a steady, but deteriorating Google traffic drop and i am struggling to find the real reasons. The site is in the video entertainment niche. Here is some data regarding the site in general: Site redesign happened around June We are not doing a lot of off-site and link building in 2013, besides social activities and content distribution to partner sites (top links from OSE are gained long time ago) GWT crawls errors are clean (besides 404s) No manual actions from Google or penalty notifications Serious fluctuations in indexed content in June, but no real effect on traffic Removal of old content during August Top keywords are dropping (may be caused by category pages removal and flattening site structure) Robots file disallows only search-generated pages No sitemap errors or warnings 2K of duplicate meta-titles and descs Thank you and appreciate your help. wyOyoA9
Technical SEO | | dimicos1 -
Follow up to Archive of Content
This is a follow up to the Question I ask: http://www.seomoz.org/q/archive-of-content I have decided that I am going to move the articles from example.com (non-commercial) to website.com (commercial) however I was having a think, some of the articles on example.com and ranking well for some keywords, maybe getting around 20,000 visits from natural search, would it be possible when moving this article just to do a 301 redirect from the page with the article example.com to the new website? Hope that makes some sense. Kind Regards,
Technical SEO | | Paul780 -
Internal Linking Structure - help Req'd
I have a website that due to the way in which it was put together a few years back always redirects to a /subdomain folder when the top level domain is entered. When analysing the new SERPS tool i spotted that when the .com domain was assessed it didn't pick up the internal links that were pointing to the /subdomain. Q) Could the /redirect cause a problem when crawled by Google, and if i'm linking back to the homepage should i be using the domain or the subdomain as the link (even though one redirects to the other......)
Technical SEO | | NSJ780 -
No follow syntax
is ref=nofollow the same as rel=nofollow? in other words, does ref=nofollow not pass any link juice?
Technical SEO | | SoulSurfer80 -
Duplicate exact match domains flagged by google - need help reinclusion
Okay I admit, I've been naughty....I have 270+ domains that are all exact match for city+keyword and have built tons of back links to all of them. I reaped the benefits....and now google has found my duplicate templates and flagged them all down. Question is, how to get the reincluded quickly? Do you guys think converting a site to a basic wordpress template and then simply using 275 different templates and begging applying each site manually would do it, or do you recommend. 1. create a unique site template for each site 2. create unique content any other advice for getting reincluded? Aside from owning up and saying, "hey i used the same template for all the sites, and I have created new templates and unique content, so please let me back".
Technical SEO | | ilyaelbert3