Hi Gianluca,

Your comment made me doubt my research. I started a new question about it. Do you have a minut to give your vision on my situation? I would really appreciate it.

https://moz.com/community/q/duplicated-content-multi-language-regional-websites

Best regards,

Bob

Thanks for the link, didn't know about that. Could indeed be game-changing!

Hi Don,

Thanks for the clear explanation. I always though disallow in robots.txt would give a sort of map to Google (at the start of a site crawl) with the pages on the site that shouldn’t be crawled. So he therefore didn’t have to “check the locked cars”.

If I understand you correctly, google checks the robots.txt with every single page load?

That could definitely explain high number of crawled pages per day.

Thanks a lot!

Hi Don,

You're right about the sitemap, noted it on the to do list!

Your point about nofollow is intersting. Isn't excluding in robots.txt giving the same result?

Before we went on with the robots.txt we didn't implant nofollow because we didn't want any linkjuice to pass away. Since we got robots.txt I assume this doesn’t matter anymore since Google won’t crawl those pages anyway.

Best regards,

Bob

Hi Don,

Just wanted to add a quick note: your input made go through the indexation state of the website again which was worse than I through it was. I will take some steps to get this resolved, thanks!

Would love to hear your input about the number of crawled pages.

Best regards,

Bob

Hello Don,

Thanks for your advice. What would your advice be if the main goal would be the reduction of crawled pages per day? I think we got the right pages in the index and the old duplicates are mostly deindexed. At this point I’m mostly worried about Google spending it’s crawlbudget on the right pages. Somehow it still crawls 40.000 pages per day while we only got around 1000 pages that should be crawled. Looking at the current setup (with almost everything excluded though robots.txt) I can’t think of pages it does crawl to reach the 40k. And 40 times a day sounds like way to many crawled pages for a normal webshop.

Hope to hear from you!

Hi Miriam,

I added keywords to the business titles of Google My Business listings and I didn’t see any big ranking improvements on categories the business already appeared at.

I added keywords to the business titles of Google My Business listings and I saw a big improvement on categories the business didn’t already appeared on. So adding “SEO” in the business title of an webdevelopment firm could result in them being shown on keywords like “SEO The Hague”.

I Removed keywords from (compeditors) business titles and didn’t see a drop in their ranking visibility.

At this point I believe keywords in the business title could help Google associate your business with a keyword or service type but isn’t (or is only a small) rankingfactor. It’s the difference between being shown, or not being shown at all. Not the difference between position 2 and 3.

Note: I only tracked this with a few Google my business pages for a short period of time.

About the advice from the Google support staff, heard it the last time on 10 dec. 2015.

Hi Miriam and Kristen,

I didn't do a big case study or what so ever on this topic but I reported this kind of keywords in the business title through Google Maps and I didn’t see any noticeable changes after the changes were applied. At least not with the top 3 in the local pack. I can however confirm that adding this information causes fluctuations since I did some testing with a few websites. Although It didn’t cause a big boost it did help a tremendous amount with being shown on certain keywords. Since there isn’t a category for every niche this helped some business being showed on the right keywords on the first place.

This is a subject were a bigger case study will be needed I think.

Btw, another interesting aspect I found is the local business support desk recommending Google plus activity on the account. Views, followers, posts etc. Did get this tips 2 times after I called them a few times. Not sure it works, but we’re trying this out for our own account at the moment.

Hi Guys,

I'm working on a project (premium-hookahs.nl) where I stumble upon a situation I can’t address. Attached is a screenshot of the crawled pages in Search Console.

History:

Doing to technical difficulties this webshop didn’t always no index filterpages resulting in thousands of duplicated pages. In reality this webshops has less than 1000 individual pages. At this point we took the following steps to result this:

1. Noindex filterpages.
2. Exclude those filterspages in Search Console and robots.txt.
3. Canonical the filterpages to the relevant categoriepages.

This however didn’t result in Google crawling less pages. Although the implementation wasn’t always sound (technical problems during updates) I’m sure this setup has been the same for the last two weeks. Personally I expected a drop of crawled pages but they are still sky high. Can’t imagine Google visits this site 40 times a day.

To complicate the situation:

We’re running an experiment to gain positions on around 250 long term searches. A few filters will be indexed (size, color, number of hoses and flavors) and three of them can be combined. This results in around 250 extra pages. Meta titles, descriptions, h1 and texts are unique as well.

Questions:

1. -          Excluding in robots.txt should result in Google not crawling those pages right?
2. -          Is this number of crawled pages normal for a website with around 1000 unique pages?
3. -          What am I missing?

BxlESTT

Hi Anirban,

As far as I know they don’t crawl websites that way. Did you check the following?

• Make sure there are no internal links (you already said so, but double check this since one link can get a whole lot of pages indexed).
• These pages don’t have external links pointing to them
• These pages aren’t in the sitemap

If you wish to keep pages out of the index you can try the following:

• Add the noindex tag
• Exclude them through your robot.txt file

I hope this helps.

Thank you both! This really helps.