Welcome to the Q&A Forum

Hi Mozzers,

Under Mobile Usability, in Google Search Console, I am seeing very few website pages getting analysed - 10 out of 40 static pages, on the website in question.

Is this to be expected or does this indicated an indexing problem on mobile?

Hi there - I am planning out an SEO migration but this thought just occured to me:

If the links into a site's previous URL went to the non-canonical version of the domain name - e.g.

to: https://theguardian.com/uk and not the correct version of that URL, which is: https://www.theguardian.com/uk

Then, if I do a redirect simply from the correct canonical version of the domain: 
https://www.theguardian.com/uk - rather than the versions of the domain that are being pointed to by backlinks - e.g. https://theguardian.com/uk - then the migration will not be carrying across all the linkjuice from the previous site.

So how would you suggest dealing with this issue?

Hi all,

I'm looking at how to handle backlinks on a site, and am seeking a tool into which I can manually paste backlinks - is there a good backlink audit tool that offers this functionality?

Please let me know! Thanks in advance, Luke

Hi Steve and thanks for the feedback - it would definitely be interesting to check - I can't imagine this is a huge issue on uncomplicated sites without thousands of pages, but who knows... testing is needed.  All the best, Luke

I often see commentators mentioning out of balance site structures/IA but what does this actually mean in SEO terms?

For example, Yoast advises: "If one category grows much larger than others, your site’s pyramid could be thrown off balance." Neil Patel advises "Try to balance the amount of subcategories within each category. Basically, try to keep it approximately even. If one main category has fourteen subcategories, while another main category has only three subcategories, this could become a little unbalanced."

Does this have any direct influence on SEO (crawlability, etc.) or is this more a UX issue? I look forward to receiving your feedback.

Many thanks Miriam - I will have a good read of that

thanks Dmitrii - It always seems to make things simpler if devs sticks to my guidance on JS - ensuring all links (inc menus) and all static text is visible whether JS is switched on or off - sadly, the often miss this bit!

When I am working on a new website I usually spec that the main navigation should work whether JS is on, or off.

I always assumed everyone did that - until today - spent a couple of hours analysing menus on websites and noticed many didn't function when JS was disabled - particularly the menus designed for mobile devices.

Any thoughts on this from fellow Mozzers would be welcome.

I have noticed that some e-commerce sites don't worry aout their store working when JS is switched off - yet some do - are there any SEO implications of losing faceted navigation/filtering functionality when JS is disabled

I tried M&S - didn't work - but Tesco did - when JS is disabled.

Hi - I am seeking an onsite search engine that is SEO friendly - which do you recommend?

And has anyone tried doofinder.com - that specific search engine - if you have, is it well aligned/attuned to the SEO aspects of your site?

Thanks as ever, Luke

Many thanks for your helpful suggestion

Thanks Casey - really useful

Hi - I am about to analyse and then re-plan the structure of a website and think it would be best to do it graphically - in the form of a chart.

Are there any tools you would recommend to visualise the structure of an existing website (perhaps something that can scan and then represent a websites) - or plan out a new/revised website?

Thanks in advance, Luke

Hi, I'm wondering which are the best Keyword research tools that provide specific volumes and suggestions re: Voice Search - including on question type searches? Any suggestions would be brilliant - thanks in advance, Luke

Thanks Tomas and Mike - good advice - I have done that and found legacy stuff they've since moved away from - there is indeed no current use for the directives.

I wonder whether there's any resource on the web that lists all robots.txt directives - and interprets them - if not then perhaps it would an idea for Moz?

Hello Mozzers - I have come across the two directives above in a robots.txt file of a website - the web dev isn't sure what they meant although he implemented robots.txt - I think just legacy stuff that nobody has analysed for years - I vaguely recall sr means search request but can't remember.

If any of you know what these directives do, then please let me know.

Thanks for your help Ramon!

Thanks Roman - that is helpful - with Yoast - do they redirect from one sitemap to another sitemap? Is that acceptable to Google (the redirect from one sitemap URL to another sitemap URL?).

Hi Mozzers - regarding the URL of a website's main website:

http://example.com/sitemap.xml is the normal way of doing it but would it matter if I varied this to:

http://example.com/mainsitemapxml.xml or similar?

I can't imagine it would matter but I have never moved away from the former before - and one of my clients doesn't want to format the URL in that way.

What the client is doing is actually quite interesting - they have the main sitemap: http://example.com/sitemap.xml - that redirects to the sitemap file which is http://example.com/sitemap (with no xml extension) - might that redirect and missing xml extension the redirected to sitemap cause an issue? Never come across such a setup before.

Thanks in advance for your feedback - Luke

Hi - I was just wondering whether a URL with extra trailing slashes should actuall redirect to the version without the extra trailing slashes... e.g. www.domainname.com/folder////// should automatically resolve to www.domainname.com/folder/ - what is your opinion on this?

Hi Andy - yes, just capitals and non-standard characters in the URLs (bad practice, basically, for URLs) - I don't think an example will help beyond what I've told you - hope that's OK, Luke

I see this strangely formatted image URLs on websites time and again - is this an issue - I imagine it isn't best practice but does it make any difference to SEO?

Thanks in advance, Luke

Hello all,

I am planning to change the title tags throughout a site and am vaguely aware (perhaps wrongly!) that changing title tags across a site is a risk factor - can be a spam flag if changes (to a specific title tag) are implemented too regularly, for example.

Would you change title tags across a site in one go, or implement changes gradually - to avoid any risk of upsetting Google. Do you have any insights/tips on the implementation of title tag changes?

Hello,

I am looking at a site on which they haven't 301'd http to https, so each URL is there whether you have http or https at the beginning.

Why would a site owner not 301 to https? Is there any logical reason not to use 301?

This particular website is simply using a canonical tag to point to the https version of each URL.

I was wondering how old the 404 data from Google Search Console actually is? Does anyone know over what kind of timespan their site 404s data is compiled over?

How long do the 404s tend to take to disappear from the Google Search Console, once they are fixed?

Thanks Logan - much appreciated, as ever - that really helps - if I was to add another * to **Allow: /?resultspage= > so **Allow: /?*resultspage= - what would happen then? ****

Thanks Logan - much appreciated - the aim would be to prevent bots crawling any parameter'd URL but only in the products section, and not all of them - see below.

I noticed the shirt URLs can be produce many pages of results - e.g. if you look for a type of shirt you can get up to 20 pages of results - the resulting URLs also feature a ?

So you end up with - for example - /shirts/?resultspage=01 and then /shirts/?resultspage=02 or shirts/navy/?resultspage=01 and /shirts/navy/?resultspage=02 - and so on - and it would be good to index them somehow. So I wonder how I can override disallow parameters robots.txt instruction only for specific paths and even individual pages?

I suppose the nub of the disagreement is this: would Disallow: /shirts/?* block /shirts/?minprice=10&maxprice=20 and also block URLS further down the URL directory structure - e.g. /shirts/mens/navyblue/?minprice=10&maxprice=20 ?

Thanks Logan -  the lead website developer was assuming that this wildcard: Disallow: /shirts/?* would block URLs including a ? within this directory, and all the subdirectories of this directory that included a “?”

If I amended the URL to 
/shirts/?minprice=10&maxprice=20 would robots.txt work as intended right there?

and would that robots.txt work as intended further down the directory structure of the URLs? E.g.
/shirts**/golden/**?minprice=10&maxprice=20

Thanks for your feedback Kristina - I ended up rediscovering addshoppers in this instance, which I think I first discovered via Moz quite a while back

Hi – the lead website developer was assuming that this wildcard: Disallow: /shirts/?* would block URLs including a ? within this directory, and all the subdirectories of this directory that included a “?”

The second developer suggested that this wildcard would only block URLs featuring a ? that come immediately after /shirts/ - for example: /shirts?minprice=10&maxprice=20 BUT argued that this robots.txt directive would not block URLS featuring a ? in sub directories - e.g. /shirts/blue?mprice=100&maxp=20

So which of the developers is correct?

Beyond that, I assumed that the ? should feature a * on each side of it – for example - /? - to work as intended above? Am I correct in assuming that?

I used to enjoy MozPerks before it was "sunsetted" - even though it isn't running anymore, would it be possible to have a list of the companies who participated - some were way off my radar and very relevant to specific campaigns. Perhaps Moz should simply list an unendorsed list of 3rd party tools that may be useful to members? I for one would find such a list incredibly useful.

Thanks Steve and Andy - very helpful indeed

Hi - I'm looking at a site using JavaScript dropdown navigation - Google can crawl the whole site but my thinking is this - If I ensure the dropdown navigation is functioning fully when JS is switched off, I may facilitate the search engine bots?

At the moment I can't get any dropdown effect if I turn JS off on the site but if I look at a cached page (text version) the dropdown links are visible and working.

I am wondering whether any crawl benefit is there if you take this a step further and ensure the drop downs are actually visible and working when JS is switched off?

I would welcome your thoughts on this.

Thanks in advance, Luke - 07966 729775

I've been having endless conversations about this over the last few days and in conclusion I agree with everything you say - thanks for your excellent advice. On this particular site next/prev was not set up correctly, so I'm working on that right now.

Yes I agree totally - some wise words of caution - thanks.

thanks for the feedback - it is Umbraco.

What's your experience of using robots meta tag v robots.txt when it comes to a stand alone solution to prevent Google indexing?

I am pretty sure robots meta tag is more reliable - going on own experiences, I have never experience any probs with robots meta tags but plenty with robots.txt as a stand alone solution.

Thanks in advance, Luke

Hello Mozzers - I am looking at a site that deals with URLs that generate parameters (sadly unavoidable in the case of this website, with the resource they have available - none for redevelopment) - they deal with the URLs that include parameters with *robots.txt - e.g. Disallow: /red-wines/? **

Beyond that, they userel=canonical on every PAGINATED parameter page[such as https://wine****.com/red-wines/?region=rhone&minprice=10&pIndex=2] in search results.**

I have never used this method on paginated "product results" pages - Surely this is the incorrect use of canonical because these parameter pages are not simply duplicates of the main /red-wines/ page? - perhaps they are using it in case the robots.txt directive isn't followed, as sometimes it isn't - to guard against the indexing of some of the parameter pages???

I note that Rand Fishkin has commented: "“a rel=canonical directive on paginated results pointing back to the top page in an attempt to flow link juice to that URL, because “you'll either misdirect the engines into thinking you have only a single page of results or convince them that your directives aren't worth following (as they find clearly unique content on those pages).” **- yet I see this time again on ecommerce sites, on paginated result - any idea why? **

Now the way I'd deal with this is:

Meta robots tags on the parameter pages I don't want indexing (nofollow, noindex - this is not duplicate content so I would nofollow but perhaps I should follow?) 
Use rel="next" and rel="prev" links on paginated pages - that should be enough.

Look forward to feedback and thanks in advance, Luke

Ignore.

Thanks Logan for your help with this - much appreciated. Really helpful!

Thanks again Logan.

What would Disallow: /?* do because that is what the site I am looking at has implemented. Perhaps it works both ways around?

I imagine it's easy to disallow the wrong thing or possibly not disallow the right thing. Ugh.

Thanks Logan - I was just reading: Disallow: /*? # block any URL that includes a ? (and thus a query string) - do you know why the ? comes before the * in this case?

Hello Mozzers - Just wondering what this robots.txt instruction means: Disallow: /french-wines/?*

Does it stop Googlebot crawling and indexing URLs in that "French Wines" folder - specifically the URLs that include a question mark?

Would it stop the crawling of deeper folders - e.g. /french-wines/rhone-region/ that include a question mark in their URL?

I think this has been done to block URLs containing query strings.

Thanks, Luke

Hello Mozzers

Would you use rel=canonical, robots.txt, or Google Webmaster Tools to stop the search engines indexing URLs that include query strings/parameters. Or perhaps a combination?

I guess it would be a good idea to stop the search engines crawling these URLs because the content they display will tend to be duplicate content  and of low value to users.

I would be tempted to use a combination of canonicalization and robots.txt for every page I do not want crawled or indexed, yet perhaps Google Webmaster Tools is the best way to go / just as effective??? And I suppose some use meta robots tags too.

Does Google take a position on being blocked from web pages.

Thanks in advance, Luke

Hello Mozzers - I am researching the most popular sites within a category - are there any services that identify the most popular sites within a category - I noticed Alexa do offer a service but I have no idea who else offers such a service (Majestic, perhaps?).

Thanks in advance, Luke

Thanks for your feedback Andy - In this instance the site has a history of penalties, so perhaps more vulnerable. It's hard to know for sure, of course.

Hi Matt - thanks for feedback - my main concern is upwards of a million affiliate backlinks that have come in recently - they are live, unfortunately.

The site is performing fine and many of the backlinks are freely given - just a few - however they have started some major affiliate marketing work and I see the backlinks coming in are nofollow and there are lots of them.

To give you an example - the latest company linking has sent through over 1,100,000 follow links to a few pages on the website (mainly a calculator for financing the motorhome).

These are all follow links and I'm thinking 302 or nofollow would be far safer, yet they're all live right now. I believe the client did ask for nofollow tags but they weren't implemented correctly.

One problem is the site is so reliant on these follow backlinks from affiliates for its rankings - if I just march in there and implement nofollows it will hit organic rankings.

I am working for a motor homes company that works with a network of dealers.

Having just analysed the site I notice that dealers are sending links to the site - lots of them. They are all follow links and are freely given. ADDED: There are upwards of a million new affiliate backlinks and then a load of pretty normal freely given backlinks with dealers who have commission arrangements, etc., with the company on motorhome sales.

Now this doesn't feel right to me because even if it isn't purposefully manipulative, it may appear so because of clear commercial relationships between my client company and the dealer businesses.

So I will recommend nofollow althought the site will lose a huge number of backlinks as a result. What are your thoughts on this?