Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Hi Angela,
Yes, those redirects should be updated. For each redirect that Google passes through, you lose about 10% of the authority from the referring page. So if you have two-step redirects like that, you'll lose ~20% of the authority. Having redirect chains like that will also negatively affect your load time.
There are about as many answers to this question as there are SEO practitioners.
All things equal on-page, the tie breaker could come down to:
These are a few examples, but it demonstrates the point that there are a lot of factors that are considered by Google when determining position.
Hi,
According to this article by Moz on hreflang, yes, having an hreflang tag with the language only will help you cast your net out to English speaking searchers from other regions.
Hi,
You'll have to track these visits by way of GA tracking parameters in the destination URL of the redirect.
Eric said basically what my next recommendation would've been. Building a new site on a new domain is not the way to handle a penalty from Google.
Having 2 websites confusing for people; i.e. people who find both of your sites through a series of refined Google searches. Additionally, it's not likely to solve your problem. Unless you moved your physical location AND you re-branded your company, Google almost certainly knows that your new site is an offshoot of your old site. Therefore, your new site is fighting an uphill battle, competing against its older self, and with Google's preconceived notions of what you're trying to accomplish.
Hi,
After reading your post, I'm wondering why the old domain hasn't already been 301 redirected to the new site? That would really be the ideal situation for not having your two sites compete against each other. You'd want to do this at the page level, so equivalent pages on the old site get redirected to the most relevant page on the new site.
For example: www.oldsite.com/calculator redirects to www.newsite.com/calculator
As with most things in SEO, it depends. Depends on the search volume of the keyword, how many sites you're competing with, how many of those sites are well optimized, how many links your page/site have, how authoritative/trustworthy your site is, and so on. There are a lot of variables in the mix here, so it's not feasible to provide a numerical answer.
Hi Neil,
If you've got two pages targeting the same keyword, they're going to compete against each other and you likely won't see either of them too high up in SERPs. I recommend canonicalizing one towards the other. Keep the one that is stronger in terms of URL structure, PA, and links (if any).
Hi Ramendra,
To my knowledge, you can only provide directives in the robots.txt file for the domain on which it lives. This goes for both http/https and www/non-www versions of domains. This is why it's important to handle all preferred domain formatting with redirects, that point to your canonicalized version. So if you want http://www to index, all other versions redirect to that.
There might be a work around of some sort, but honestly, what I described above with redirection towards preferred versions is the direction you should take. Then you can manage one robots.txt file and your indexing will align with what you want better.
THANK YOU!
This has been driving me nuts! I really don't understand why people keep calling it a 'factor'. It's a tool for determining SERPs, NOT a factor!
Factors are things that we, as webmasters/SEOs/whatevers can control. We obviously have no control over RankBrain.
I actually have come across a handful URLs that are NoIndex, I'll DM you a list once complete.
I can't be certain this is the root of the problem (I've never seen this error in the crawl report), but based on the error you said you're getting, I believe it's a great starting point.
Hi,
This sounds like it's more related to the meta robots tag, not the robots.txt file.
Try this:
Commenting so I can see any response from a Mozzer, as I'm still waiting on my 500 pt. shirt.
Hi Rena,
You technically can do that, but it's not recommended - for the exact reason you state above. More often than not, 2 sites aren't going to have the same set of disallow rules.
Additionally, you should also be using robots.txt files to direct search engines to your XML sitemap, and if you're sharing a robots file, then you can't specify 2 different sitemaps on 2 different domains.
The best way to mitigate this problem would be to update the destination URLs in your Adwords Campaigns. You can do this in bulk relatively quickly using the Adwords Editor desktop application.
Hi Steven,
You'll definitely want to apply 301 redirects to any site that you move to HTTPS. For most sites, this can typically be done with a single redirect rules that essentially replaces http with https, so you won't have to comb through each URL and apply one-to-one redirects.
No need to worry about losing link juice, Google views these types of 301s differently than a typical 301, and all authority will pass through them.
Canonical should also be applied, this will help search engines learn your new URL structure and ensure they index the new HTTPS URLs.
Cryus Shepard wrote a great post with all the necessary steps for a secure migration, check it out here: https://moz.com/blog/seo-tips-https-ssl
Good luck!
In your HTACCESS file (of the redirecting domain) where the existing redirect is located.
Yep, I've used that exact syntax for some of my clients.
Hi,
You won't be able to see that information in Google Analytics unless you tag the resolving URL in the redirect with UTM parameters. i.e. www.mywebsite.com >> www.yourwebsite.com/?utm_medium=301&utm_source=mywebsite.com
Hi Kurt,
I think it depends on the likelihood of your readership's interest in reading content on both topics. If you think there's value (and likelihood of interest) in people being able to easily access content on both sides, then go with one blog and categorize. If you think there is little-to-no chance of a research blog reader being interested in the fundraising content, and vice versa, then go with two separate blogs.
A side note, since you're concerned with SEO - while looking at your site, I noticed a lot of your meta descriptions are duplicates, and your homepage is duplicated across two different URLs. Run this search for details: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=site%3Acentiment.co
Hi,
Google will count this as duplicate content, regardless of 'affiliate' status or not. You've got a few options here:
I think option 3 is the most viable, since you won't have to ask the affiliates to make any operational changes.
Hi Chris,
Screaming Frog has been my go-to XML sitemap generator for years. Plenty of customization options for exclusions and inclusions.
Hi Ramendra,
Based on what you said, it sounds like both versions of your site exist and are indexed, and you want to mitigate your duplicate content risk. If that's accurate, here are my recommendations on this:
Hi,
You've got a really big mess on your hands, IMO. Search engines absolutely _do _penalize duplicate content, and it sounds like you have a ton of it on the existing sites, with plans to create even more.
What types of locations are the 25 different sites going to be targeting? All within the same country, or one for each of 25 different countries? The answer to this question will drive any further recommendations.
No problem! Yes, same is true for HTTP and HTTPS
That is correct, you should be using rel=next/prev for markup on paginated sections. But after noticing the www and non-www issue, I don't think your problem is related to canonicals or prev/next.
Regardless of what you're doing with canonical tags or prev/next, you pages should never be accessible at both www and non-www versions. You're going to be at a duplicate content risk as long as both versions exist.
Hi,
Regarding the first set of URLs: I took a look at a handful of those URLs, and it's entirely possible that you're getting duplicate notices on those. Rogerbot flags any 2 pages as duplicates if the source code of those URLs matches at 90% or more. So it's not identical, but not different enough that search engines can discern. Most of the products you've listed there have no content, or a very small amount, meaning that when you consider the rest of the code involved with that page, it mostly matches the homepage.
Regarding the second set: I ran those URLs through Screaming Frog and don't see any canonical tags. Keep in mind, just because URLs aren't indexed in search engines, doesn't mean Rogerbot doesn't have access to them.
*Update - on further digging, I think I found the source of all of your duplicate issues. Both www and non-www versions of your URLs are accessible. One of them should redirect to the other, doesn't matter which, but both should not render.
Hi,
It sounds like you're going down the right path. Disallow and section of the site that has personal information, as there's no value in having bots crawl that, keep them on important content longer! In addition to Checkout and Basket/Cart, you should also disallow the My Account area if your site has one.
Your next grouping, I'm assuming these are the parameters by which you pages can be sorted. If so, yes, disallow all of those, they're only going to cause duplicate content flags for you in the future. I'm not sure which CMS you are using, but some eComm platforms also have 'email to a friend' URLs that are a major source for dupes and can often be identified and disallowed by another parameter.
Hope this helps narrow it down for you!
Hi,
There are a couple things you can do to help Google serve up your preferred TLDs in each country.
First, you should identify (if you haven't already) the preferred geography in Search Console - see this link for more info on how to set that up: https://support.google.com/webmasters/answer/62399?hl=en
Next, you can add the appropriate hreflang tags to the pages on each of the different site versions. Moz has an extensive guide on how to do this here: https://moz.com/learn/seo/hreflang-tag
Hope that's helpful!
Hi,
Duplicates of this nature can only be fixed by adding canonical tags to one preferred version of that product. When you have multiple instances of the product throughout many categories, search engines aren't sure which to show, so they'll usually not show any of your results. For more on canonical tags, read this Moz article.
You cannot remove it from the Moz report, as that's part of its purpose - to show you instances of duplicate content.
It's almost impossible to say without a screenshot, but there's most like a filter applied that's causing this. Go to Admin > View > Filters and comb through the list of filters looking for one involving the 'hostname' field.
Thanks so much for reporting back! Nonetheless, the responses here helped strengthen my case that on-site elements like this should not exists solely for bots, so I'm making headway!
They will show up in Open Site Explorer, I've seen shortlink URLs show up there for a handful of clients.
Hi,
You can use bit.ly for backlinks, but keep in mind, they are 301 redirects, so you lose a bit of link juice when search engines step through them.
If you chose to do this, you'll also want to tag the URLs before you shortlink them, otherwise your Google Analytics reporting will show all that traffic as Direct/(none) in the Channel report.
Yes, I typically reserve the homepage for branded search. On most sites, the content of the homepage is too broad to really be a helpful entry point from organic non-branded search.
Heyo!
I would not recommend your homepage exists at any URL other than example.com for the following reasons:
I'm sure other could drum up more reasons, but the ones I've listed here should be enough to dissuade you.
You can typically fare much better by giving non-branded keywords interior pages that are specific to that topic rather than the broader homepage content. This will increase the likelihood of people finding what they're looking for and is a better way to tailor your content to your audience and to algorithms.
Just saw your update: Yes, only enter ^www.
I would go ahead and do that after you apply the filter. Google reporting is based on the URI string, so this won't cause any data loss.
Hi,
Take the following steps:
Enter ^www\. for the search stringGoing forward, it's actually much simpler than that, all you need to do is go to Admin > Property Settings > Default URL. Change it to the non-www version and you're good to go!
Hi,
It sounds like you might need to implement hreflang tags. Have you done so already? Without being able to specify each country-level shop in Search Console via ccTLD, this is your best bet to ensure the right pages show in the right versions of Google. Here's Moz's guide to using hreflang tags. For each of your country-specific subdirectories, you'll need to add the appropriate tag which will specify the country and language of the page. This helps keep Swiss pages out of Google.de, etc, etc.
Hi,
This isn't necessarily a problem, but XML sitemaps should be as clean as possible before they're uploaded. i.e., no 301'd URLs, no 404s, no dupes, no parameter'd URLs, no canonicalized, etc..
Are they duplicates in the sense that one has caps, and the other doesn't? As in /example.html and /Example.html. If so, you'll want to fix that.
If they're identically formatted URLs, there should be no problem, but you're at duplicate content risk if they're different in anyway and not canonicalized.
If they won't take them down, you can use the link disavow tool in Search Console. Since DA is a Moz generated metric, you might not see a change, but Google will stop 'counting' those links to your site.
Hi,
It's important to remember that your link profile isn't about quantity, it's about quality. 10,000 junk links are less valuable than 10 high quality links.
I took a quick look at your link profile in OSE, and it looks like you have some questionable sites pointing to yours. For example, the one from this domain: http://organisedlinks.com, even the domain of that site screams "link farm!". And that same domain is also part of a network, I can see multiple links to your site with the first part of the URI being "/business/business-services/", which is not a great indicator of link quality when that appears on multiple linking domains. Most of your links are from domains that all appear to being aggregating links/data from the same place.
Get rid of those links, acquire some relevant links from industry related sites and you should see your DA tick back up.
Hi,
I'm trying to read over your content and it just keeps loading more products. Very frustrating. You should consider pagination and having a default amount of products load on the page.
Regarding your internal links, you have one the says "2000 gifts for men" and one that says "shop gifts for men". These links point to the page the exist on, with a hashed URL. Links should never point to the same URL on which they live, I see this a lot and its very frustrating, it looks spammy to search engines, and is confusing/frustrating/irritating for searchers. They click that link only to be taken to the top of this long page again? Not a good user experience. There's also a link for your homepage buried in "..." in the first paragraph, that also looks spammy/cloaky to me.
I'm not saying these page load and internal linking issues are your whole problem, but they're definitely not helping your situation and the simplest things like this should be tackled first.
There are no external links pointing to this page, if your competitors have similar pages with some recently acquired good quality external links to them, this is potentially the cause of your recent decline.
Hi Rob,
Since you've got no links pointing to these pages, a 410 would be your best bet. This will get them removed from the index the quickest, and you'll start to see these errors in Search Console drop. Doing 301 redirects would also get them removed from the index, but that's also to slow down your performance, and since these pages can't be accessed other than SERPs anyway, 301s aren't going to provide much long-term value.
Here's some more info on the difference between 404s and 410s: https://searchenginewatch.com/sew/how-to/2340728/matt-cutts-on-how-google-handles-404-410-status-codes
Hi,
I never recommend URL changes for the purpose of improving rankings. While URL structure is a ranking factor, it's not a big enough one to justify restructuring your URLs for. See this article for more on how insignificant this is as a ranking signal.