Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Ah yes when it's pointed out like that, it's a conflicting signal isn't It. Makes sense in theory, but if you're setting it to noindex and then passing that on via a canonical it's probably not the best is it.
They're was link out in that thread to a discussion of people who still do that with success, but after reading that I would just use noindex only as you said. (Still prefer the no index on the robots block though)
I don't think it should be blocked by robots.txt at all. It's stopping Google from crawling the site fully. And they may even treat it negatively as they've been really clamping down on blocking folders with robots.txt lately. I've seen sites with warning in search console for: Disallow: /wp-admin
You may want to consider just using a noindex tag on those pages instead. And then also use a canonical tag that points back to the main job category page. That way Google can crawl the pages and perhaps pass all the juice back to the main job category page via the canonical. Then just make sure those junk job pages aren't in the sitemap either.
If that was the final product page, then yes, you should be using a standard htacess rewrite command to ensure that the final product urls are always www.domain.com/productx But in saying that, the way you had it is totally fine if all the other url possibilities have a canonical tag that points back to optimised version (original product url - www.domain.com/productx)
The htacess rewrite it's not something you should be handling manually. Magento has that option inbuilt into it. It would be a fair amount of work if you had to do that manually. and I would just run with the canonical option if that were the case. Any good eCommerce platform should have the inbuilt ability to automatically remove the category folders and other search queries from the final product url.
Sometimes it's ok to leave the category folders in the url, it just depends on the products being sold. Below would be an example where I would leave the category folders in the url if I was selling different colored soccer balls.
www.sports.com/soccer-balls/black-white/
www.sports.com/soccer-balls/blue-white/
www.sports.com/soccer-balls/red-yellow/
Yes it does take some patience to find a nice domain that's relevant and has stats. I wouldn't 301 the other domain to the new domain as a new domain won't be able to handle that many new inbound links which is what your effectively doing by sending a 301 with links behind it to your new domain.
If the urls follow any particular pattern then you can use a htaccess redirect and return the header code 410 / 403 / 404 to Google. (I suggest 410) They will soon drop out of the index.
I don't know the exact .htaccess syntax off the top of my head but it will be something like this:
If they all come from the same folder then it would look something like this:
RedirectMatch 410 ^/folder/.*$
If they have a common character string after the forward slash (such as xyz) then it would look something like this:
RedirectMatch 410 ^/xyz.*$
If they have any common character string footprints at all (such as xyz) then it would look something like this (now I'm guessing):
RedirectMatch 410 ^/()xyz.$
This would be a pretty easy fix if all of those spammy urls have any common characters after the forward slash or they all originate from a certain folder.
Re-purposing domains is no where near as safe or effective as it used to be. Your asking for trouble if your going to completely change the topic of the website and expect easy ranking benefits from the old off-topic inbound links. (this is one of the footprints they used to recently de-index low quality PBN's because the sites topic had completely changed after the domain was re-birthed)
In saying that, I wouldn't start with a fresh domain either.
I would do my homework on domcop or register-compass and find an expiring/expired/deleted domain with great on-topic link metrics and start the new site that way. If your niche isn't that big then you may be able to find an on-topic domain with metrics like DA40 - PA45 - TF35 - CF40 for under $500. That would give your new site a huge kick start. Hunt for decent Trust Flow with good topical relevance. If you build a great site under a domain like that your rankings will come much easier than using a fresh domain.
Those paths all seem fine as they are all legitimate ways of getting to that bunch of products. I'm also assuming that the final page on each of the below urls is a page that contains a selection of products and you can still click on an individual product from the list and go to its url.
1)www.domain.com/productx
2)www.domain.com/category/productx
3)www.domain.com/category/subcategory/productx
4)www.domain.com/bestsellers/productx
It's even debatable whether you need canonical tags on any of those above urls, it all depends on how different those pages are from each other with regards to the content on the page before the products. If all of those above urls had different H1 tags and different content before the product feed then they are all stand alone legitimate pages that don't need canonical tags. But if they're all virtually the same and not much customization has been done to the auto generated pages then yes, they should all have a canonical tag back to www.domain.com/productx perhaps, or the most suitable page.
A bigger issue I think you may have is the url of the final single product page. It shouldn't be like this:
www.domain.com/category/subcategory/productx/final-product/
or like this either:
www.domain.com/bestsellers/productx/final-product/
Optimally, it should read like this no matter how the visitor got there:
www.domain.com/final-product/
In most cases, for optimal Seo, an extension, plugin, or htaccess command should rewrite the final product url to strip out all of the category url paths or best seller url paths from the urls so your final product page url is short and clean like this: www.domain.com//final-product/ even though the path is really ths: www.domain.com/category/subcategory/productx/final-product/
It's pretty hard to tell what the optimal solution for your site is without having a look at it and understanding your product range and categorization a bit better, but I hope that helps a bit.
I've found using carefully placed all caps in certain parts of the description works great. Both the title and the description should have some type of cliff hanger or teaser at the end too. Take a look at the below example.
The SECRET to Smart META DATA is to Catch the EYE!!
Meta Data that has COMPELLING calls to ACTION & cleverly seeded BUZZ words achieves
FANTASTIC results for CTR. Great descriptions leave you hanging for...
And I always use the mofoseo snippet optimizer so I know exactly whats its going to look like in the search results (provided Google doesn't change it)
Hi James, As long as you set up your 301 redirects properly everything should be fine, even without changing any of the backlinks. The amount of authority and link juice lost via the 301 should be minimal at best, especially when the redirect is on the same root domain.
In a situation like this it's good to set your preferred www URL in webmaster tools to make things nice and clear for Google. In webmaster tools get them to fetch the new www URL and submit it for indexing again. Ranking loss is usually minimal even when using a completely different domain name.
Hello Ted, yes they can hurt your site in a number of ways. Site owners tend to make these links anchor text rich, so if you've got a link in your footer saying 'Blue Widgets' then effectively you may have 30+ anchor text links from your own site. And yes, anchor text backlink ratios are calculated with the inclusion of internal links from your own site.
As you also mentioned, these footer links are draining the juice out of your main contextual links within in your main page's copy. Effectively, those nice internal silo links you send to your inner pages are being watered down by all of your dofollow footer links.
So do I make all of my footer and menu links no-follow then? And there's the problem. You won't find definitive answers on this because it's grey hat. Google will tell you that nofollow links are links that you don't want to vouch for. So are you going to send a signal to Google that tells them you don't trust internal links on your own site because you added the nofollow attribute to them?
And yes, whether the link is nofollow or not, it's still included in your overall anchor text ratios.
Now we move into PR sculpting. Google will tell you not to do that, and that PR sculpting doesn't work anyway. Is that because it still works very well indeed? Why are there so many authoritative sites that still use the nofollow attribute on some of their internal links? Don't they trust these internal links, or are they channeling link juice to the pages they want it diverted to.
If the rest of your link profile was pretty clean, and all of your offsite SEO was above board, then I think you'd be pretty unlucky to get a penalty from internal links coming from the footer of your own site.
One of my sites is ranking top three for many medium to semi-hard keywords that uses PR sculpting. Every single menu and footer link is nofollow. So the homepage has about 30 nofollow internal links on it, and only two contextual links in the main copy that link to the other inner pages that I wanted to rank.
That site has remained top 3 for over 1.5 years now without a hitch. This definitely isn't conclusive evidence by any means. The site itself is very strong and has great content too, but it seems as though all of the nofollow links haven't affected it negatively. And the inner pages that I sent all of the juice to are ranking #1 too. In my opinion PR sculpting does work but I also think it's dicey.
In your situation, I would maybe just dial down the exact match anchor text and change them to partial match links. Google do devalue your internal footer links to a certain degree but there's no black and white answer. If your site is big, then your generating 100's of anchor text links, and although they're devalued, it's still a bit dicey.
Hello Ted, yes they can hurt your site in a number of ways. Site owners tend to make these links anchor text rich, so if you've got a link in your footer saying 'Blue Widgets' then effectively you may have 30+ anchor text links from your own site. And yes, anchor text backlink ratios are calculated with the inclusion of internal links from your own site.
As you also mentioned, these footer links are draining the juice out of your main contextual links within in your main page's copy. Effectively, those nice internal silo links you send to your inner pages are being watered down by all of your dofollow footer links.
So do I make all of my footer and menu links no-follow then? And there's the problem. You won't find definitive answers on this because it's grey hat. Google will tell you that nofollow links are links that you don't want to vouch for. So are you going to send a signal to Google that tells them you don't trust internal links on your own site because you added the nofollow attribute to them?
And yes, whether the link is nofollow or not, it's still included in your overall anchor text ratios.
Now we move into PR sculpting. Google will tell you not to do that, and that PR sculpting doesn't work anyway. Is that because it still works very well indeed? Why are there so many authoritative sites that still use the nofollow attribute on some of their internal links? Don't they trust these internal links, or are they channeling link juice to the pages they want it diverted to.
If the rest of your link profile was pretty clean, and all of your offsite SEO was above board, then I think you'd be pretty unlucky to get a penalty from internal links coming from the footer of your own site.
One of my sites is ranking top three for many medium to semi-hard keywords that uses PR sculpting. Every single menu and footer link is nofollow. So the homepage has about 30 nofollow internal links on it, and only two contextual links in the main copy that link to the other inner pages that I wanted to rank.
That site has remained top 3 for over 1.5 years now without a hitch. This definitely isn't conclusive evidence by any means. The site itself is very strong and has great content too, but it seems as though all of the nofollow links haven't affected it negatively. And the inner pages that I sent all of the juice to are ranking #1 too. In my opinion PR sculpting does work but I also think it's dicey.
In your situation, I would maybe just dial down the exact match anchor text and change them to partial match links. Google do devalue your internal footer links to a certain degree but there's no black and white answer. If your site is big, then your generating 100's of anchor text links, and although they're devalued, it's still a bit dicey.
I've found using carefully placed all caps in certain parts of the description works great. Both the title and the description should have some type of cliff hanger or teaser at the end too. Take a look at the below example.
The SECRET to Smart META DATA is to Catch the EYE!!
Meta Data that has COMPELLING calls to ACTION & cleverly seeded BUZZ words achieves
FANTASTIC results for CTR. Great descriptions leave you hanging for...
And I always use the mofoseo snippet optimizer so I know exactly whats its going to look like in the search results (provided Google doesn't change it)
Those paths all seem fine as they are all legitimate ways of getting to that bunch of products. I'm also assuming that the final page on each of the below urls is a page that contains a selection of products and you can still click on an individual product from the list and go to its url.
1)www.domain.com/productx
2)www.domain.com/category/productx
3)www.domain.com/category/subcategory/productx
4)www.domain.com/bestsellers/productx
It's even debatable whether you need canonical tags on any of those above urls, it all depends on how different those pages are from each other with regards to the content on the page before the products. If all of those above urls had different H1 tags and different content before the product feed then they are all stand alone legitimate pages that don't need canonical tags. But if they're all virtually the same and not much customization has been done to the auto generated pages then yes, they should all have a canonical tag back to www.domain.com/productx perhaps, or the most suitable page.
A bigger issue I think you may have is the url of the final single product page. It shouldn't be like this:
www.domain.com/category/subcategory/productx/final-product/
or like this either:
www.domain.com/bestsellers/productx/final-product/
Optimally, it should read like this no matter how the visitor got there:
www.domain.com/final-product/
In most cases, for optimal Seo, an extension, plugin, or htaccess command should rewrite the final product url to strip out all of the category url paths or best seller url paths from the urls so your final product page url is short and clean like this: www.domain.com//final-product/ even though the path is really ths: www.domain.com/category/subcategory/productx/final-product/
It's pretty hard to tell what the optimal solution for your site is without having a look at it and understanding your product range and categorization a bit better, but I hope that helps a bit.
Re-purposing domains is no where near as safe or effective as it used to be. Your asking for trouble if your going to completely change the topic of the website and expect easy ranking benefits from the old off-topic inbound links. (this is one of the footprints they used to recently de-index low quality PBN's because the sites topic had completely changed after the domain was re-birthed)
In saying that, I wouldn't start with a fresh domain either.
I would do my homework on domcop or register-compass and find an expiring/expired/deleted domain with great on-topic link metrics and start the new site that way. If your niche isn't that big then you may be able to find an on-topic domain with metrics like DA40 - PA45 - TF35 - CF40 for under $500. That would give your new site a huge kick start. Hunt for decent Trust Flow with good topical relevance. If you build a great site under a domain like that your rankings will come much easier than using a fresh domain.
If that was the final product page, then yes, you should be using a standard htacess rewrite command to ensure that the final product urls are always www.domain.com/productx But in saying that, the way you had it is totally fine if all the other url possibilities have a canonical tag that points back to optimised version (original product url - www.domain.com/productx)
The htacess rewrite it's not something you should be handling manually. Magento has that option inbuilt into it. It would be a fair amount of work if you had to do that manually. and I would just run with the canonical option if that were the case. Any good eCommerce platform should have the inbuilt ability to automatically remove the category folders and other search queries from the final product url.
Sometimes it's ok to leave the category folders in the url, it just depends on the products being sold. Below would be an example where I would leave the category folders in the url if I was selling different colored soccer balls.
www.sports.com/soccer-balls/black-white/
www.sports.com/soccer-balls/blue-white/
www.sports.com/soccer-balls/red-yellow/
If the urls follow any particular pattern then you can use a htaccess redirect and return the header code 410 / 403 / 404 to Google. (I suggest 410) They will soon drop out of the index.
I don't know the exact .htaccess syntax off the top of my head but it will be something like this:
If they all come from the same folder then it would look something like this:
RedirectMatch 410 ^/folder/.*$
If they have a common character string after the forward slash (such as xyz) then it would look something like this:
RedirectMatch 410 ^/xyz.*$
If they have any common character string footprints at all (such as xyz) then it would look something like this (now I'm guessing):
RedirectMatch 410 ^/()xyz.$
This would be a pretty easy fix if all of those spammy urls have any common characters after the forward slash or they all originate from a certain folder.
I don't think it should be blocked by robots.txt at all. It's stopping Google from crawling the site fully. And they may even treat it negatively as they've been really clamping down on blocking folders with robots.txt lately. I've seen sites with warning in search console for: Disallow: /wp-admin
You may want to consider just using a noindex tag on those pages instead. And then also use a canonical tag that points back to the main job category page. That way Google can crawl the pages and perhaps pass all the juice back to the main job category page via the canonical. Then just make sure those junk job pages aren't in the sitemap either.
Ah yes when it's pointed out like that, it's a conflicting signal isn't It. Makes sense in theory, but if you're setting it to noindex and then passing that on via a canonical it's probably not the best is it.
They're was link out in that thread to a discussion of people who still do that with success, but after reading that I would just use noindex only as you said. (Still prefer the no index on the robots block though)