Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:
User-agent: ia_archiver
Disallow: /
This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.
More information about this can be found here: https://archive.org/about/exclude.php
I agree with your SEO agency.
Heading tags should be used to provide a hierarchy for the content on a page, not just to format text.
Heading tags should be avoided in page templates where they are not relevant in demonstrating the hierarchy of content on a page - ie. in the header, main navigation, sidebars and footer.
I find it strange that this happens with Yoast as it points the canonical tag for the homepage to a 301 redirect URL. I don't think it's a big problem as search engines seem to handle it well, but it's a problem that I wouldn't expect to see in such a popular and reputable SEO plugin.
If you use the plugin editor to remove the trailing slash you will remove the trailing slash from all pages of your site (something you may not want to do).
If you only want to remove the trailing slash from the homepage the easiest option would be to manually add the canonical URL without the trailing slash in the Advanced tab for Yoast when editing the homepage (as Linda said above). Doing this for one page will take you 2 seconds and fix the problem!
Hi Drew,
If you have multiple canonical tags on a single page, Google will ignore both of them so you will definitely need to to remove the old canonical tag so there is only one on each page.
But it sounds like you are trying to use canonical tags correctly to resolve these duplicate content issues.
If you wan't to read more about canonical tags, this recent article by Patrick Stox on Search Engine Land is excellent: http://searchengineland.com/canonical-tags-easy-right-whats-worst-happen-274635
Cheers,
David
Hi Miss Thumann,
Definitely writing your own content is the best option here.
You won't get any value out of republishing content from high DA sites and it could even have negative consequences.
If you want to publish content from another site because you think your readers might like it, you can do it, BUT you should use a canonical tag to tell search engines who the original source is.
Cheers,
David
Hi William,
Moz would be picking up these URLs because they are linked internally, and they are being reported as duplicates because these pages are identical and there are no canonical tags in place.
https://im.tapclicks.com/signup.php should have a self referring canonical tag so that any query strings added to the end of the URL are canonicalized back to https://im.tapclicks.com/signup.php
Adding this canonical tag will resolve these 'High Priority' issues
Cheers,
David
Hi 94501,
I went through this about 6 months ago with a big Australian company. I did pretty much everything you suggested and the transition was very smooth.
Here are my dot point on what I think you should do and answers to your questions:
Only other thing I would add is to make sure you keep things like page titles, heading tags, etc., the same for the pages that are already performing well.
Also, if the new site is using a template that has big changes from the old site, make sure you do all the standard checks to make sure it's 'SEO-friendly' so you don't run into any issues not directly related with moving content to a new domain.
Overall, goal should be to keep the old and new pages as similar as possible - unless you are making improvements!
Cheers,
David
Sounds like they are wanting to make turtlesforsale.com/turtles-for-sale/ their homepage, which is quite strange. "Keyword stuffing" URLs like this will not help SEO in any way. I would advise against it and leave the homepage as turtlesforsale.com.
eg. for other pages on the site: turtlesforsale.com/state/city/ is better than turlesforsale.com/turtles-for-sale-in-state/turtles-for-sale-in-city/
Like most things in SEO, think of what is better for humans. The first example is much cleaner and easier to read, and for SEO purposes, there would be no difference between the two if all other factors were the same.
A similar question was asked (and answered very well) here: http://moz.com/community/q/url-seo-better-directory-structure-vs-exact-keyword-phrase
Hi Chris,
I quickly ran your site through Screaming Frog and found that this new page is not linking from your site anywhere - it's what's known as an "orphan page".
By not linking from the site, it is not receiving any "authority" from your domain and you will have a hard time getting it to rank for anything.
The first thing I would do is to make sure it's linking from your site and check how it's going in a few weeks.
Cheers,
David
Hi Luke,
The onsite SEO for both sites actually looks pretty good - looks like Moz has helped you optimize the pages well!
Where both sites are not doing so well appears to be with the links pointing to them.
On the Document Management Software site, there really aren't any good links pointing to the site. It looks like there are a few really old businesses linking to it, but that's about it. Not enough "quality" links for it to be competing with the top sites for "document management software".
On the physio site, it looks like there are a lot of old "SEO links" pointing to the site which would be doing more harm than good (hopefully these are disavowed), eg:
http://sportsinjurycentre.net/sports-physiotherapy-kippax-expert-sports-injury-management/
http://diagnosisneckpain.com/neck-ache-sufferers-talk-to-a-deakin-physio-for-fast-relief/
In summary, onsite SEO is looking good but I think you should focus your efforts on improving the offsite SEO for both sites.
Cheers,
David
Hi Alex,
I think you are talking about featured snippets here, not rich snippets.
There is no structured data markup that will help you get featured snippets. Google uses an algorithm to determine which page it thinks answers the search query the best and pulls through a snippet from that page into the SERPs.
If you are not familiar with featured snippets and how to optimize your content to increase your chances of getting them, I would suggest reading/listening to the following resources:
I also wrote an in-depth guide on how to find easy featured snippet opportunities for you website, which might help you.
Cheers,
David
Hi Luke,
I don't think you have anything to worry about with making the improvements in one go.
As long as you are improving your titles to make them more relevant to the content on the page (and not just jamming them keywords), you won't be upsetting Google and you would only see positive results.
Cheers,
David
Hi jacleaves,
The problem here is with your canonical tags (more info on canonical tags here).
This page: http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
Has a canonical tag pointing to: http://amishdirectplaysets.com/playset-details/
The same thing is happening on all of your product pages.
I would change all of your product pages to use "SEO-friendly" URLs and have a self-referring canonical tag.
Eg. your example page: http://amishdirectplaysets.com/playset-details/?mfgID=A-1&catID=4
Would become: http://amishdirectplaysets.com/playset-details/a-1-standard-climber-vinyl-playset
And the canonical tag would point to: http://amishdirectplaysets.com/playset-details/a-1-standard-climber-vinyl-playset
Cheers,
David
Hi Renia,
I'm not sure what they mean when they say "I wonder if MOZ is not reading the infinite scrolling pages the way that Google does"...
Pagination (rel="next" and rel="prev") and canonical tags are not in place so each page is being indexed individually, and would be causing some issues, as Moz reports. Without these tags in place, Google will not understand that the pages are a series, as your web team suggests.
You can see that they are indexed by doing a Google search for site:https://www.whiteheadindustrial.com/c-104-abrasives.aspx
Or site:whiteheadindustrial.com inurl:pagenum to see that there are over 2,000 in total of these pages indexed.
You could easily overcome this issue by getting the developers to implement pagination and canonical tags on these types of pages.
Your second issue, I think, will always be reported as duplicates in Moz because they use the same overall page template (which is taken into consideration when assessing duplicate content) and there is little unique content on them - this is common on e-commerce sites when there isn't much product information available or products are very similar.
However, I don't think you need to worry about this too much. Google is pretty generous when it comes to product pages like this and you shouldn't be negatively impacted by it.
Your third issue with title tags should be resolved once pagination and canonical tags are implemented correctly throughout the site.
Hope this helps 
Cheers,
David
If your pages are correctly marked up with Schema you don't don't need to submit them to anything and even if it marked up correctly, it's still up to Google to decide if they give you the rich snippets, or not.
ie. Simply having the correct markup does not guarantee you the rich snippets!
Hi sdaily,
I'd definitely keep the existing GA property so you can see the full history in that one property. That way, you'll still be able to compare y/y traffic, etc.
With a new property, you'll be starting your data from scratch and comparing different date ranges will be more difficult.
Just create an annotation so you can keep track of the changeover date.
Cheers,
David
Using the rewrite rule is fine.
There is no issue with domain.com/non-existant-page.html 301 redirecting to domain.com/non-existant-page which would then return a 404 status. If you go to anydomain.com/any-non-existing-page you will return a 404 page but this endless amount of possible pages will not be crawled by search engines because they do not exist and you (or anyone else) should not be linking to them.
With your old, legit 404 error pages, I would recommend 301 redirecting them to a relevant new page, a relevant category page, or the homepage if nothing else is suitable.
Hi Francisco,
I think you're focus is a little bit misdirected here.
To see the real impact of your backlinks, I would be checking Google Analytics for increases in organic traffic or Search Analytics data in Google Search Console.
Don't put too much focus on DA and don't be surprised to see it go up and down with different indexes (and don't freak out when it goes down).
Overall, my advice would be to focus on real metrics rather than third party metrics.
Cheers,
David
To add to this, you will also probably never get star ratings to appear for your homepage. I've never seen star ratings appear on a homepage but would love to see any proof of it ever happening!
John Mueller said back in 2014 that "review rich snippets should only be used in cases where "the main topic of the page needs to be about a specific product or service." -- using them on your homepage in general wouldn't be like that."
Article on SE Roundtable here.
I'd think you'd have a much better chance of getting them to display in SERPs if you moved the ratings markup to relevant service pages on your site.
Cheers,
David
Yes, you can link to other sites in text (anchor text will be the linking text) wherever you like, but if you are using their content you should use a canonical tag to acknowledge them as the original source.
What you've submitted is your sitemap for human visitors - not a sitemap for search engines.
The sitemap that you submit to Webmaster Tools for search engines will be a .xml file. Eg. instead of https://www.zenory.co.nz/sitemap it would be https://www.zenory.co.nz/sitemap.xml
There are a few ways you can create a sitemap.xml for your site. You can use a program like Screaming Frog to crawl the site and generate a static sitemap. That is, a sitemap file that won't update automatically when you add new pages or posts - you will have to generate a new sitemap every time you add new content.
Depending on which CMS you are using you should be able to generate your sitemap.xml dynamically. That is, a sitemap file that updates itself as you add new content, which makes things a lot easier - especially on large sites!
You can find out more about XML Sitemaps and see a sample here: http://www.web-site-map.com/
It is important that you only include URLs in your sitemap that you want search engines to crawl. Don't include any pages that are no indexed or blocked by robots.txt.
Hi John,
For visualizing internal link data, I think Sitebulb is the only tool that does it automatically.
You can do it with Screaming Frog data but it's a bit of a process to visualize it (check out this guide if you're interested).
Cheers,
David
There was a great case study released by Reboot Online Marketing last week on exactly this topic: https://www.rebootonline.com/blog/hidden-text-experiment/
Basically, sites with visible text performed better; text in a textarea was treated as visible; and sites with text 'hidden' with CSS and JavaScript 'Read More' buttons didn't perform as well.
As Logan says above, Google is looking to give hidden content full weight with the release of the mobile first algorithm, but this algorithm hasn't been released yet and is likely not to be released this year: https://www.seroundtable.com/google-mobile-first-index-2017-23663.html
Hope this helps.
Cheers,
David
Just to add a bit more information to this, Screaming Frog in default 'Spider' mode will identify old URLs and their status code only if it picks up links to the old URLs in the crawl (in which case you should update all internally redirected links to point to their new location), but if the old URLs are now 'orphaned', you won't find them in the Screaming Frog crawl report.
BUT, if you have a list of all of your old URLs (hopefully you ran a crawl before changing the site over!) you can use Screaming Frog in 'List' mode to find the status code of all of your old URLs and easily identify anything that needs fixing.
Cheers,
David
The canonical tag should point to the URL of the site that you want indexed.
If you want denenapoints.com to be the indexed site the canonical tag on denenapoints.com must point to itself: http://www.denenapoints.com/
NOT http://houston-injury-lawyers.com/
I would 301 redirect the http://houston-injury-lawyers.com/ domain to prevent possible duplicate issues.
Hi Ben,
It sounds like you've recently moved to https or added www. to your URLs.
The reason data in your existing Search Console account fell of a cliff is because that specific version of the URL is no longer getting any impressions in Google search.
You will need to create and verify a new Search Console account for the URL that is now being used: https://www.worldelitesoccer.com
This new account will show all the data you are looking for.
One other thing, you should also setup 301 redirects from:
https://worldelitesoccer.com
http://worldelitesoccer.com
to
https://www.worldelitesoccer.com
Cheers,
David
Hi Monu,
It's most likely because Moz hasn't crawled your shortened URLs before - anything that Moz hasn't crawled will report as PA 1.
But don't be too concerned about PA and how long it takes Moz to crawl your links!
And don't think that going through a URL shortener is going to "boost" the link value because it's reported as a higher DA/PA link - it doesn't quite work like that
Cheers,
David
Hi 1stOTLJK,
I own a site that's in Google News and have worked on getting more dozens of client's sites into Google News.
While it's not mentioned in Google's News guidelines anymore, I believe that you still need to have at least 3 regular authors publishing news content on the site. How many writers does the site currently have?
You also need to make sure that the content you are submitting to G News is actually newsworthy. If the site posts basic blog content like "10 tips to improve X" or "10 best X for you", this needs to separated from what you are submitting to G News.
If you have to, create a 'News' category on the site and only add newsworthy articles to this category. Then, when submitting to G News, only submit this category.
Eg. submit yoursite.com/news/ instead of yoursite.com
And don't worry about being knocked back with your submission attempts - it won't leave a "stain" on the site that will prevent you from being successful with a future submission.
If you can PM me the site, I'd be happy to take a quick look at it and provide you with some more specific tips.
Cheers,
David
Hi Tim,
I agree with Laura on the canonical tags. I've worked on several large Magento sites and I've never seen any issue with the way Magento handles it - by canonicalizing product URLs to the root directory.
In fact, I actually prefer this was over assigning a product to a 'primary' category and using that as the canonical.
As Laura said, a reduction in the total number of indexed pages might actually be a really big positive here! More pages indexed does not mean it's better. If they are low quality/duplicate pages that have been removed from index, that's a really good thing.
I did find some issues with your robots.txt file:
Solution: I would leave Disallow: /? in robots.txt because it stops a product filter URLs being crawled, but I would add the following line:
Allow: */?p=
This line will allow your paginated pages to be crawled, which will also allow products linked from those pages to be crawled.
Hope this helps!
Cheers,
David
Hi Inevo,
Canonical tags are being used correctly and it doesn't actually look like any of the URLs with query strings are indexed in Google.
I'm going to go off the topic of canonicals now, but still related to the crawl and index of the site:
Has the site changed CMS in the last year or two? It's possible that some of the 400k URLs indexed are old or were not canonicalized properly at some point in time, so they were indexed.
The problem with how the site it currently setup is that it is basically impossible for search engines to crawl because of the product filter. I wrote an article about this a while ago (link), specifically to do with product filters in Magento. Product filters can turn your site into a 'black hole' for search engines - which is definitely happening in this case (try crawling it with Screaming Frog).
I'd recommend blocking product filter URLs from being crawled so that search engines are only crawling important pages on the site.
You should be able to fix this be adding these 3 lines to your Robots.txt:
Disallow: *?
Disallow: *+
Allow: *?p=
(Note: please check that you don't need to add more parameters to Allow)
These changes will make crawling your site much more efficient - from millions of crawlable URLs, to probably 30-35k.
Let me know how this goes for you
Cheers,
David
Hi Richard,
Alarm bells go off when bounce rate is that low and it's usually because there is more than one GA tracking code on on your pages.
Looking at the source code on your homepage, I can see that there are 3 tracking codes on the page! You will need to remove two of these so you are left with only one tracking code on each page if your site. You will then get more accurate data.
Cheers,
David
Hi David,
I have to disagree with Dmitrii a bit here.
I would say the answer to your question depends on the size of your site and how many articles and tags you are talking about - on bigger sites with more tags, they can become quite an issue.
On a big-ish news site that I own and run (14,000+ articles), I added "noindex" to all tag pages (2,000+) and then blocked them in robots.txt a few months later (once Google had crawled and removed them all from index).
The reason for removing and blocking them for search engines is that tag pages generally don't offer anything unique or of substantial value when compared to other pages on your site site, and they can also use up a lot of your "crawl budget".
For the site mentioned, I saw around a 30% organic traffic increase after implementing these changes.
The tags are still there for users, but they are not visible to search engines and means that search engines are now only crawling pages that we want indexed and ranking, instead of wasting their time crawling thousands of links to thousands of tag pages.
Basically, don't think of tags as a way to "help your SEO" because they probably won't and they have the potential to do the opposite.
If you think creating a particular tag will help you get long-tail traffic, write an article about that topic instead! I guarantee you it will perform a lot better than a tag page.
Cheers,
David
Hi Luke,
For that site, it looks like some pages are resolving to https and some are not.
It's possible that they are in the middle of a transition to https and are rolling it out slowly over the entire site - it's hard to know.
Asking why a companies wouldn't automatically redirect http to https - again, it's hard to say. It might be that they are also in the middle of a transition to https, or it might be that they simply don't know any better/are using an SEO service that doesn't know any better.
Google will probably be able to figure it out and it shouldn't really hurt them, but if we want things to be perfect and want to make things easier for search engines, then the then URLs should definitely resolve to https, if it's available.
Cheers,
David
Hi Jamie,
It sounds like the dev site has been removed from index, which is good, but I would check two things straight away on the proper site:
If this is a WordPress site, check Settings > Reading and make sure that the box that says 'Discourage search engines from indexing this site' is unchecked.
If you aren't sure how to do any of this, send me a PM with the domain names and I'd be happy to take a quick look.
Cheers,
David
Hi Matt,
Just to clarify what Martin said, adding alt text to your images won't actually help your images get indexed, but they will help improve your rankings in image search - so they are two separate things.
If you do an image "site:" search, you'll see that 40+ of your images have been indexed: https://www.google.com.au/search?q=site:bayareahomebirth.org&source=lnms&tbm=isch&sa=X&ved=0ahUKEwj5_LfJ0qDUAhVIlJQKHQeFCVIQ_AUICigB&biw=1280&bih=590&gws_rd=cr&ei=HCEyWemYKcib0gS3jIzYBw
It's only been two weeks, so the rest of the images may just take a little bit longer.
I can also see that a lot of your images are not hosted on your domain (they're hosted with Squarespace).
If you are looking at indexed images in your Search Console account, these images will not be reported as indexed in your SC account because they are not hosted on the same SC domain - but they are actually indexed.
You can check to see if an image is indexed by pasting the image URL into image search, eg. https://www.google.com.au/search?q=https://static1.squarespace.com/static/56b65664b09f955cb8aed482/t/56ba99d5555986520338de96/1455069653711/big%2Bpoo%2Bbig%2Bbath.jpg&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjZnOif1aDUAhXFH5QKHT0PDrcQ_AUIBigB&biw=1280&bih=590
Overall, I don't think you have any issues with images being indexed - just reporting accuracy issues because some of them aren't hosted on your domain.
Cheers,
David
Hi Lesley,
I think Google would be pretty good at handling things like this on e-commerce sites, so I wouldn't worry about it (as long as those page elements are plain text and not using heading tags, for some reason).
Also, google:off/on tags are only used by Google Search Appliance, and are ignored by Google Bot. You will still have content within off/on tags indexed in normal Google search.
Cheers,
David
If you want to leave a legitimate blog comment to engage with the author and the community, go for it!
If you're only leaving blog comments to get links to "help your SEO" - don't do it.
It's spam and it won't help you at all.
Hi Ricky,
Rand did a great Whiteboard Friday on 'How to Optimize for Competitors' Branded Keywords' in February this year that I think you'll find helpful
Both options you suggested could be successful.
I think it really comes down to two things:
If they have no issue with openly targeting competitor brand names, I'd say do it on the current site. This will also be the cheaper option.
If they want it to appear as an anonymous product comparison, obviously, you will have to build a separate site - which will require more budget.
You don't need to worry about a Google penalty with either of these options.
Cheers,
David
Hi Becky,
The solution to your problem is to remove the hreflang tags from the canonicalized product pages.
Doing this will leave only your canonical URLs specifying hreflang - with these pages also having the self-referring canonical tag in place.
Leaving the canonical tag in place on the duplicate product pages will continue to solve your duplicate content issues - no need to specify hreflang there.
Hope this helps!
Cheers,
David
Hi Luke,
Don't worry too much about your DA.
Have you lost rankings? Have you lost organic traffic? If the answer is "no" to both questions, you're doing fine.
DA will fluctuate with each update because there are different volumes of links crawled by Moz each time. Maybe they crawled less links pointing to your domain or maybe some sites have stopped linking to you.
If you haven't lost any links, you might see your DA increase in the next Moz update.
If you have lost links, try to reclaim those links or create new links to replace them.
Hopefully you'll see an upwards trend in your DA over time, but don't be surprised if it's a bit up and down along the way - and don't stress if it happens to be down. Focus on more important metrics like organic traffic and keyword rankings.
Cheers,
David
Hi Julian,
The reason your GSC account isn't reporting your images as indexed is that they are on a different subdomain to your GSC account - GSC will only report indexed URLs that are on the exact subdomain of that account.
And are the images actually used on the site? None of them showed up in a Screaming Frog crawl...
Cheers,
David
Hi bridhard8,
In your example, "SHOP NIKE SHOES" is the anchor text because it's the text between the <a>tags.</a>
<a>Change that button text and you change the anchor text.
Cheers,
David</a>
Hi SEO Analytics,
You can start understanding how Google ranks websites by reading Moz's Beginner's Guide to SEO.
Once you've finished reading it, read it again.
Cheers,
David
Hi Eddy,
Edit: this was already answered before I could post my reply. But I've left the example.
The issue with the meta robots tag is that you are using curly quotation marks around robots and noindex:
You have:
“robots**” content=“noindex”/>
Instead of:
name="robots" content="noindex"**/>
This will fix your issue.
Cheers,
David
Hi Jeff,
You can use hreflang to target the same language in different regions.
Google provide a good example of this about halfway down this page: https://support.google.com/webmasters/answer/189077?hl=en
Like in the example here, you may want to use the US version of the site as the English default and specify regions for the rest.
Cheers,
David
Hi GTAMP,
I would definitely remove the h2 tag from your site-wide header tagline.
Heading tags should be used to demonstrate a hierarchy to the content on a page and using a h2 in the site's tagline in the header is not doing this.
I wouldn't expect to see any noticeable gains from only removing the h2 tag, but it's one of those things where it all adds up in the end.
Cheers,
David
Hi Alessia,
While you could say these are duplicates (as Igor said), I don't think Moz should be reporting these as duplicates because they are not an issue as they are handled with hreflang and pagination - and won't actually cause you any problems.
However, I would update your Italian pages to use Italian titles. They seem to be in Italian everywhere except the title - this will be important for you to get Italian traffic to your site.
There is also a small issue with your hreflang implementation where you are only specifying a language. You must specify a language and a country for hreflang to validate.
Cheers,
David
Hi Gianluca,
He was talking about a WordPress website which, by default, already has a 301 redirect in place on the homepage URL with the slash to the URL without the slash.
What's strange is that the Yoast SEO plugin points the homepage canonical tag to the URL with the slash - which is 301 redirected to the URL without!
I know search engines handle this very well and probably just ignore the canonical tag, but if you want it to be 100% correct (and why wouldn't you?) adding a the canonical tag in the Advanced tab is a very easy fix.
