I've had a sudden a increase in crawl issues as of yesterday (like 300 from a steady 10, does anyone else have this issue?
-
the main issue is that it's now indexing both www and http:// - anyone else got this issue or had any changes suddenly on their crawl results?
-
If it makes things easier, based on your settings in that campaign you've always been crawling "all sites," including http://domain.com and http://www.domain.com. If this is due to a redirect, I'd guess that it's due to a redirect having been removed.
Have there been any changes made on that site recently?
-
Thank you both. I was wondering why it had done this now (the domain is at least 4 years old) but i'm going to do what you say and hope it fixes so thanks again!
-
My first response was going to be just what Ryan suggested!
I would look into why both are being indexed, which could be as simple as needing to redirect one version to the other, and then change your preferences in GWT & Bing. The issue should sort itself out after that has been taken care of.
-
Hi Rebecca. Do you have a 301 redirect in place to set your preferred domain? i.e. If someone goes to http:// are they redirected to the www, vice versa, or do both work? If both work you could get crawling on both versions. Ideally you'll pick your preferred setup, add the 301, and also set that same preference within GWT and Bing.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
No index and Crawl Budget
Hello, If we noindex pages, will it improve crawl budget ? For example pages like these - https://x-z.com/2012/10/
Technical SEO | | Johnroger
https://x-y.com/2012/06/
https://x-y.com/2013/03/
https://x-y.com/2019/10/
https://x-y.com/2019/08/ Should we delete/redirect such pages ? Thanks0 -
How do you 'close down' a website?
Hello all, If a company acquires a smaller company and 'absorbs' its products and services into its own website, what is the protocol with closing down the smaller company's site? So far we added our branding to the site alerting their visitors to the imminent takeover, and 301 redirected certain pages - soon we'll be redirecting all the pages to their counterparts on the main website. Once that's done, should we noindex the old site? Anything else? Thanks, Caro
Technical SEO | | Caro-O0 -
Does a GTLD extension 'count' as part of the target keyword?
Hopefully someone can shed some light on this for me. Reading about GTLDs, I came across this quote from TSO Host: 'What we don’t know is whether an extension can double up as a keyword, which is picked up by Google and treated identically to the rest of a domain name. I.e. - would ‘bristolguitars.music’ have more ranking potential than ‘bristolguitars.com’ as ‘music’ is a relevant search word?' Source: https://www.tsohost.com/blog/how-do-new-gtlds-affect-seo Does anyone know if a GTLD extension does double up as a keyword? For example, if Nike buys 'Nike.shoes', does this double as the keyword 'Nike shoes', or is Google and other search engines just looking at the domain name _before _the GTLD extension? I'm looking at .photography for examples (not my niche) and seeing folks are having mixed results ranking for 'Keyword + Photography', so would be keen to hear your thoughts.
Technical SEO | | ecommercebc0 -
Duplicate Page Title Crawl Error Issue
In the last crawl for on of our client websites the duplicate page title and page content numbers were very high. They are reading every page twice. http://www.barefootparadisevacations.com and http://barefootparadisevacations.com are being read as two different pages with the same page title. After the last crawl I used our built in redirect tool to redirect the urls, but the most recent crawl showed the same issue. Is this issue really hurting our rankings and if so, any suggestions on a fix for the problem? Thank you!
Technical SEO | | LoveMyPugs0 -
Does anyone use buzzfeed to creat links traffic and increase brand
Hi i would like to know if anyone uses http://www.buzzfeed.com to create links, gain traffic and increase brand awareness. I have signed up for an account but cannot get it to work and would like some help. I can get my content on there but cannot manage to get the links to work I signed up for this account a while back and a friend shown me how to use it but i have forgotten. here is my page http://www.buzzfeed.com/lifestylemagazine some links work and some links do not. what i am trying to do is to publish stories from my site as well as other sites and have the link included where you press the title and it goes to the site any help would be great
Technical SEO | | ClaireH-1848860 -
What's the best way to solve this sites duplicate content issues?
Hi, The site is www.expressgolf.co.uk and is an e-commerce website with lots of categories and brands. I'm trying to achieve one single unique URL for each category / brand page to avoid duplicate content and to get the correct URL's indexed. Currently it looks like this... Main URL http://www.expressgolf.co.uk/shop/clothing/galvin-green Different Versions http://www.expressgolf.co.uk/shop/clothing/galvin-green/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/1 http://www.expressgolf.co.uk/shop/clothing/galvin-green/2 http://www.expressgolf.co.uk/shop/clothing/galvin-green/3 http://www.expressgolf.co.uk/shop/clothing/galvin-green/4 http://www.expressgolf.co.uk/shop/clothing/galvin-green/all http://www.expressgolf.co.uk/shop/clothing/galvin-green/1/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/2/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/3/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/4/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/all/ Firstly, what is the best course of action to make all versions point to the main URL and keep them from being indexed - Canonical Tag, NOINDEX or block them in robots? Secondly, do I just need to 301 the (/) from all URL's to the non (/) URL's ? I'm sure this question has been answered but I was having trouble coming to a solution for this one site. Cheers, Paul
Technical SEO | | paulmalin0 -
404-like content
A site that I look after is having lots of soft 404 responses for pages that are not 404 at all but unique content pages. the following page is an example: http://www.professionalindemnitynow.com/medical-malpractice-insurance-clinics This page returns a 200 response code, has unique content, but is not getting indexed. Any ideas? To add further information that may well impact your answer, let me explain how this "classic ASP" website performs the SEO Friendly url mapping: All pages within the custom CMS have a unique ID which are referenced with an ?intID=xx parameter. The custom 404.asp file receives a request, looks up the ID to find matching content in the CMS, and then server.transfers the visitor to the correct page. Like I said, the response codes are setup correctly, as far as Firebug can tell me. any thoughts would be most appreciated.
Technical SEO | | eseyo20 -
301 Redirect Issue
I'm having an issue with 301 redirects: Let's see if I can verbalize my thoughts on this one... So we just recently moved our site to Wordpress. One of our new 301 commands is redirecting oursite.com/news to oursite.com/blog . However there are other links from our previous site that look like oursite.com/news/XYZ and the issue is that, because wordpress structures its links differently, that URL is not equivalent to oursite.com/blog/XYZ. Instead, it might look something more like oursite.com/blog/yaddayadda/XYZ or something. Does that make sense? The issue is that when I find an old link of ours on google that looks something like "oursite.com/news/XYZ" or "oursite.com/news/ABC" it is automatically replacing "news" with "blog". When I try to go in manually and redirect anything that says "/news/XYZ" to "/blog/yaddayadda/XYZ" it still doesn't work. It still just replaces "news" with "blog." Wow I realize that might not make sense to anyone but if it does - please advise!! Thanks!!!!
Technical SEO | | EntrustSEO0