Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Yes, that is what I was referring to regarding Search Console.
It's _possible _that Moz isn't recognizing the H1 because it's below the H2 tag and there are two H1 tags. Those two issues should be resolved whether or not it's the cause of Moz not registering your H1.
Hi Tiffany,
Have you rendered the page in Google Search Console to see how Google will render the site? In most cases, this will answer any question you have about Google's perception of a page element.
As Johnny said, you definitely want to qualify your keyword more. Let's say for the sake of discussion that this link was the magic bullet that got you to rank #1 for "towing" - presumably it's a local service so if you rank that well for such a generic term, you would get a lot of traffic sure, but 99.99% of it's going to be very unqualified.
The effort required to do this is relatively high and you'll see zero impact - actually, maybe a decline since you'd be redirecting all your current URLs. Google doesn't care about trailing slashes or file extensions on URLs. The most important thing is that URL variants all resolve to a single address. For example https://australiatimenow.com.au/category/health/ should always resolve instead of https://australiatimenow.com.au/category/health
Canonicals will help with the trailing slash issue, but that's not the sole reason for your traffic loss. Read up on Angular and address those issues first. It's definitely going to be a boatload of work, but the impact you'll see from that will far outweigh any smaller optimizations you're able to make.
Google has stated more than once that accordioned content will be crawled, indexed, and weighted the same as any other content on the page. This wasn't always the case, but as mobile usage grew and accordions became commonplace for good mobile UX, they pivoted their stance on this type of content. In your example, I would especially not be worried about it since the primary content of the page is in the Overview section.
It was wise to kill one of those pages. If I were in this position, I would use as much data as possible to inform the decision. I would look at:
Google Search Console for impressions, average position, CTR
Google Analytics: entrances, page value, bounces
You see that because the site is built to be a progressive web app, as seen in their Lighthouse Report.
When did the redesign occur? I looked at the wayback machine and it looks like it was well over a year ago. It's concerning that you're still seeing rank/traffic issues.
I noticed that MozBar isn't able to grab a status code from the site, which is concerning. From the looks of things, you're using a version of Angular that isn't Google-friendly. When I click links on the site, none of the data in MozBar changes, which means Google likely isn't seeing anything different when they follow links either, effectively making your site a single page. Read more here: https://www.searchenginejournal.com/angular-seo-guide/303849/#close
You should read Google's Search Quality Evaluator Guidelines. This document includes details of what they call E-A-T (Expertise, Authority, Trustworthiness). It's a long read and relatively dry, but there is some really good material in there that will help you better understand how Google evaluates a page. DA and PA are much simpler metrics than what Google actually uses in their algorithm, they're meant to help you make decisions and gauge competition, but these numbers are not provided by or used by Google.
And for what it's worth, I've seen countless instances of sites with great content and no links outrank mediocre content with a decent link profile. As Google gets more advanced in their ability to understand the content (using NLP), searcher intent, and engagement signals, links become less and less important.
I feel like I’ve read this comment somewhere before....
Seriously, Moz. It’s time to shut this thread down.
Moz team, looks like this has turned into feeble attempts at creating links instead if discussing them. Might be time to turn comments off.
Has anyone ever seen a log file analysis return only images and no actual page URLs?
David,
This type of link building has been dead for a long time. I'd definitely recommend spending your time elsewhere. Your time will be better spent creating great content.
Taylor,
If you've already shown your boss data from a Google employee and that didn't work, nothing provided here will be convincing either.
I've been in similar situations where my boss has a different stance and the way we've always settled it is by running a test. This gives you your own data to prove your case.
Take a subset of pages on your site, split it in half and try your way and his way. Be careful in what you choose so it's not skewed one way or another; look at search volume for target keywords, current traffic volume, etc.
Hi Sharon,
Your site should be entirely one or the other - and since it sounds like you've already got an SSL, I'd go with HTTPS. If you have traffic bouncing back and forth between HTTP and HTTPS, you're resetting the session in Google Analytics when someone clicks from an HTTPS page to a HTTP page. This creates a ton of dirty data and lost attribution.
Hi Jon,
If your objective is to prevent Moz from reporting crawl issues from these file types, you don't want to nonidex/nofollow them, you simply want to prevent Moz's User Agent (rogerbot) from crawling these file types.
Add these 3 lines to the top of your robots.txt file:
User-agent: rogerbot
Disallow: /*.dxf$
Disallow: /*.dwg$
I took a look at the paginated versions of these pages and it looks like canonicals and rel next/prev are all setup correctly. Have you gone into Search Console and specified the purpose of the 'start' parameter? That might help Google better interpret what's going on. Typically pagination is handled by a more accurately named parameter which could be causing confusion, despite having canonical/prev/next tags configured properly. Under the Crawl menu in Search Console, go to URL Parameters and configure 'start' for pagination.
In the situation outline by the OP, these pages are noindexed. There’s no value to clutterig up crawl reports on these pages. Block rogerbot from non-critical parts of your site, unless you want to be alerted of issues, then don’t.
Did your name change? Or did I pull Patrice out of nowhere?!? haha
Anyway, if it's product detail pages you're referring to, you don't have too much to worry about. According to John Mueller of Google (https://www.seroundtable.com/google-rewriting-product-descriptions-22254.html) duplicates at the product level aren't a big deal. However, if you have any way to differentiate your pages, I would recommend doing that. Can you add a product review section? Q&A? Anything like that to create content unique to _your _versions of the products would be beneficial.
Patrice,
Any time this question comes up, it always raises a larger question of why? What is the purpose of 2 different domains if they contain the same content? It's almost always inadvisable to do this, but there are a few use cases where it's necessary. If you could provide some more details on the why, I can help answer your question a little better.
Alternatively, if you don't have a solid "why" or it doesn't come to you immediately, you might consider scrapping one of the domains 
I agree, unfortunately Moz doesn't have an internal disallow feature that gives you the option to feed them info on where rogerbot can and can't go. I haven't come across any issues with this approach, crawl depth by search engine bots will not be affected since the user-agent is specified.
Hi,
I would advise against this for a couple reasons. First of all, dropping and re-adding pages to the index isn't quick, it often takes weeks before Google will obey a noindex tag. Second, even if it were quick, is going to cause problems at the authority level - is your site a trustworthy source of information? This week, yes, next week, no. It's best to have long-term content that can build trust/authority over time without ephemeral.
An alternative approach you might take is writing content about the particular trials offered, this will help prevent thin content. You might also consider adding a call to action on empty pages that prompt users to provide an email address and you'll notify them when trials of type XYZ have opened back up.
I second what Gaston said. This usage of robots.txt is one of the most common misconceptions in SEO, so don't feel bad. Google actually explicitly says to not use robots.txt for index-prevention in their webmaster guide.
To add to Gaston's point, make sure you remove the robots.txt disallow when you add the meta noindex tag he provided. If you don't let them crawl the page, they won't see the tag.
You can also use remove these URLs temporarily in Search Console by going to the Google Index menu and selecting "Remove URLs". It'll remove from search results, then when they come back to crawl that page again (as long as you're letting them), they'll see your noindex tag and keep it out.
Wanted to jump back in on this topic as I've just confirmed my initial suspicion.
I just added a new client to our Moz account and had the exact same issue, crawler unable to access the robots.txt file. It's a secure site and was configured in Moz without the HTTPS. When I go to the robots.txt file without https://www, it redirects to the same thing as yours where the / between the TLD and page path gets removed.
Reconfigure your site and it should begin to work.
Hi there,
There's something odd going on when I try to access your robots.txt file without the www. The www gets added back on, but when it does, the slash between the TLD and page path gets deleted, see below. I'm guessing your domain in Moz is configured without the www, which means RogerBot is getting redirected to this slash-less version of the file.
Hi Zack,
Have you configured your parameters in Search Console? Looks like you've got your prev/next tags nailed down, so there's not much else you need to do. It's evident to search engines that these types of dupes are not spammy in nature, so you're not running a risk of getting dinged.
Hi,
Patience, young padawan, it's only been a week
It's basically impossible to redesign a website without some organic traffic loss. There's a lot for search engines to consider, everything they previously knew about your website is gone, they have to learn your new site architecture, URL structures, content, meta data, etc. etc.
It sounds like you covered all the most important bases and got rid of some old school tactics. Give it some time, you should start to see a rebound in a few weeks. The worst thing you can do is panic and try to fix a bunch of stuff that isn't actually broken.
Hi,
I wouldn't recommend doing this, it's going to be a lot of work and you probably won't see any benefit. The historical value of those URLs is much stronger than any boost you'd get by taking out a few date folders. It's very common for blogs to have this URL structure, search engines are well aware of this structure and aren't going to ding you for having long or deep folder URLs.
Additionally, since your blog is 10 years old, you've probably got tons of posts, which means tons of 301 redirects. You have to take Google with a grain of salt when they come out and say things like "301s pass all authority now", it doesn't mean you should go out and redirect every page on your site. At some point you're going to begin affecting page speed, particularly on mobile devices.
This question only leads me to followup questions...
Hi Johny,
I recommend only putting schema markup on actual blog posts, and not the category pages. Chances are pretty good that you're not getting much organic traffic to those pages anyway.. There's not a lot of reason for someone to search for something that would land them there, so much so that on every blog I manage I've had /category/ and /tag/ pages deindexed.
However, if you're still wanting to do this, you'd have to set the markup for category pages as a separate tag in GTM and then apply a trigger that only applies to URLs in the category folder.
Hi there,
I haven't heard of using product markup on category pages until reading your question. It's a good idea and while researching I came across this other thread from a couple years ago: https://moz.com/community/q/google-rich-snippets-in-e-commerce-category-pages
My main concern is that most ecomm category pages have a lot of products showing by default, and if you've got a snippet of JSON+LD for each of those, you're going to hurt your page speeds. Just something to be mindful of and monitor if you do decide to go this route.
Hi Nichole,
You're correct in noindexing these pages, they serve little to no value from an SEO perspective. Moz is always going to alert you of noindex tags when they find them since it's such a critical issue if that tag shows up in unexpected places. If you want to remove these issues from your crawl report, add the following directive to your robots.txt file, this will prevent Moz from crawling these URLs and therefore reporting on them:
User-agent: rogerbot
Disallow: /tag/
Disallow: /category/
*edit - do not prevent all user-agents from crawling these URLs, as it will prevent search engines from seeing your noindex tag, they can't obey what they aren't permitted to see. If you want, once all tag & category pages have been removed from the index, you can update your robots.txt to remove the rogerbot directive and add the disallows for tag & category to the * user agent.
Hi,
I'd definitely recommend keeping users on the product detail page to read reviews, for both structured data and UX reasons.
You need to have reviews on the page if you want the rating to show up in search results, there might be a way around this but I wouldn't trust it in the long-term.
From a UX perspective, if people can easily read the reviews on the same page, they'll have a lot less reason to do a search for "Product A reviews" and you're not distracting them by sending them to a different page. People have very short attention spans, so keep them on the product detail page, they shouldn't have to navigate back to the product in order to complete a purchase.
Having reviews built into the page also helps different your pages from each other (in the event you have two very similar products) as well as the competition.
Eric,
Add the following lines to your robots.txt file and you should be good to go. This will prevent Moz (aka RogerBot) from crawling and reporting on URLs that are tagged with UTMs:
User-agent: rogerbot
Disallow: /*?utm
You should also add self-referring canonical tags on these pages, which will prevent multiple versions of the same page from being indexed. For example:
http://www.yoursite.com/some-page?utm_source=facebook&utm_medium=social should have a canonical tag that looks like this:
Matthew,
For branded search, this box is generated using the information from your Google My Business listing. Do you have that listing claimed, verified, and accurate?
Hi Roman,
It's very difficult to say how your site will be impacted by adding schema markup. What I'd recommend doing is adding a notation in Google Analytics for when the schema was added to the site and do before & after analysis. I'd recommend looking at all your typical Google Analytics metrics, i.e. organic traffic, site engagement (did the schema attract more qualified users?), and any other metrics you focus on. In addition to GA analysis, I would also run a before & after analysis in Search Console, where you get pre-click metrics like impressions, clicks, average position. If you've got a rank tracking tool, I'd check there too for any noticeable improvements.
Hope that's helpful!
Hi there,
This is a great question, and one that I addressed almost identically to what Anthony recommended in a Moz post I wrote about a year ago: https://moz.com/blog/heres-how-to-keep-301-redirects-from-ruining-your-seo
However, one of the comments on that post suggested a better solution than UTMs that I've been using ever since, which is basically to just invent your own parameters. Here's what my parameters look like now:
/old-page >>> /new-page?redir=301&src=/old-page
The problem with UTMs is that they override the original source/medium information. The new format allows you to retain that attribution data while still being able to use the parameters for isolating this traffic.
Hi,
It sounds like you're trying to rank the homepage for something other than branded search. For most sites, the homepage should really only rank for branded keywords. Using interior pages is a much better approach for broad, non-branded keywords. Here are a few reasons why you'd want to rank interior pages:
*edit- I also noticed you have some odd usage of a noindex tag on the homepage, I'd nix that. I don't think it's accomplishing the intended goal, and could be creating some confusion for search engines.
Isaac,
This is the same issue that I mentioned a fix for on this thread: https://moz.com/community/q/should-i-be-worried-about-our-duplicate-content
If you canonicalize product A to product B, product A will eventually drop out of search engines' index, not what you want to happen. Read through the recommendations I made on the other thread.
The directive that is literally "Disallow: /" will prevent crawling of all pages on your site, since technically, all page paths begin with a slash. Robots.txt files can only live at the root folder (subdirectory) of a site, so if you want to disallow a folder, you'll need to specify that with a directive like "Disallow: /folder-name/
Nice! This is exactly what the first part of my recommendation was.
Hi Isaac,
This question pops up a lot in the forum. Here's what's happening: Moz flags any two pages on a site with 90% or higher source code match as duplicates. When you account for the code that makes up your template (header/footer/styles/etc.) that's often the bulk of the source code of a particular page. In the examples you provided, the product details and descriptions aren't robust enough to make a distinguished difference for Moz.
What you can do to handle this is write unique descriptions for each product, add more specs & details, a Q&A section, or even customer reviews. While this seems like a lot of effort for a duplicate content issue that probably isn't that big of a deal, everything listed there will vastly help your user experience and drive them closer to purchase.
Hope that helps!
Hi there,
There's two things that should be done to fix this:
The first is that www and non-www versions should not both be available, one version should redirect to the other version. It doesn't matter which you choose, but in your case, Google already has www indexed, so I'd go with that. This will take care of the first two examples of dupes (www.joeborders.com vs. joeborders.com).
The second is the canonical tag, assuming you go with the www version of your domain, your canonical tag would look like this: . This would take care of the /index issue.
You said this site was built using Ruby of Rails, that seems like overkill for a basic content site (unless there's more to it that I'm missing). You're probably overpaying for a solution that far too robust for what you need, so yes, you might consider searching for a new developer and hosting solution.
Even though your product titles have lower search volume, you still want to use your product detail pages as the preferred ranking URL for any product-specific query. This is where the benefit of long-tail keywords comes into play, you'll get a lot less traffic from them, but the quality (likelihood of them converting/purchasing) is much higher.
Take the 'Nicolai – 8th Wonder finger bit – Granite' for example. If I've done a Google search for that, my research is already done and I know exactly what I need. If I click on a result that takes me to a category page, that's not going to be as useful to me. But if the search result is for the product detail page, I'm landing on the exact page I want. It's got all the product info & specs I need, pricing, and most importantly, an Add to Cart button.
Hope that's helpful. For more info on ecomm SEO, I'd recommend taking a look at back through some of the Moz posts on the subject: https://moz.com/blog/category/e-commerce
Jeroen,
In addition to helping mitigate duplicate content, the hreflang tag you're referring to helps keep content in its respective version of Google. For example, you don't want English content showing up in Google.es (Spain), or German content showing in Google.co.uk.
How is the content broken out on the site, in different subfolders, subdomains, or TLD?
Marco,
Do not proceed with that task, it is not the proper way to use a canonical tag. If you put a canonical tag pointing a product to a category page, the product URL will eventually get removed from the index and therefore won't drive any traffic for product-specific queries.
You mentioned "24 different products quite similar but not exactly the same", is Moz flagging them as duplicates? If so, I'd recommend differentiating these products more. You could write more robust descriptions or add user-generated content such as reviews or Q&A.
Google is indexing non-www, so I'd recommend sticking with that version. Similar to what you've done with the HTTP > HTTPS redirect, you should set up a redirect rule so send all www > non-www.
It doesn't matter which you go with in the end (www or non-www), the important thing is you choose one version and stick with it.
Becky,
The parameters that denote the page are going to make this a bit more difficult, but what you should do is markup the pages with rel next/prev tags. You can read more on that here: https://moz.com/blog/seo-guide-to-google-webmaster-recommendations-for-pagination
You'll also want to configure the parameters in Google Search Console. Under the Crawl menu on the left, the last option is URL Parameters. Chances are the page parameter already shows up here, but you can use this feature of GSC to specify what the parameter does. Just click Edit in the furthest right column and choose 'Yes: Changes, reorders, or narrows page content' and then 'Paginates'. Next, just select which URLs with that parameter you want Google to crawl and you're all set.
You are not at risk for being removed from Google's search index. In fact, Google has recently adjusted their stance on hidden content and no longer ding sites for having it **IF **it's done for the benefit of UX, specifically mobile device UX. Here's a recent statement from Google's Gary Illyes on the subject: https://www.seroundtable.com/google-content-tabs-hidden-change-22950.html
Review the layout and functionality of these pages, if you believe it's helpful for UX, then stick with it. You could also run a test where you take a sample of your category pages and unhide the content by default to see if they perform better.