I can't agree more with Gary, we probably need some more information to know what kind of black hat you're possibily dealing with. One of the first things I tend to look at trying to find out if the site is using some ways of black hat tactics are:

• Backlink profile, if the quality of links is low or certain percentages between follow/ nofollow links are different then it could be a sign.
• Look at the site with Google as a user agent and see if the site is showing different information then to a real user.
• Just do a select all on the site to see if they hide any content (yup, still happens).

Hi Meaghan,

What you could do is use a rank tracker and add your 50 keywords to it. If you're able to export them it could tell you what kind of pages are already ranking for the right search terms. In the old days I would have used Bradleys option and have gone with Google Analytics. But as it won't tell you the keywords anymore it's really hard to make a good decision based on that lack of data.

Like Matt says, could it be that there is a robots.txt file that could block these URLs? Could it be that your XML sitemap generator is using cached results for your site. Can the pages be found on the page via links on other pages?

I thought that value was a bit outdated, turns out to be still accepted. Although it probably only address this issue for him in Google and I assume it will still remain one in other search engines.

Besides that the problem offered a way better solution in allowing Google not on the HTTPS site.

Did you mean a noindex tags for robots or a specific one for googlebot? With the second one I probably get the downvotes.

You're welcome, it was mostly due to noticing that the first snippet, the homepage, had no snippet and the rest of the pages did have one. That led me to looking at their URL structure. Good luck fixing it!

Hi Phillipp,

You almost got me with this one, but it's fairly simple. In your question you're pointing at the robots.txt of your HTTP page. But it's mostly your HTTP**S **pages that are indexed and if you look at that robots.txt file it's pretty clear why these pages are indexed: https://www1.swisscom.ch/robots.txt all the pages that are indexed match with one of your Allow statements are the complete Disallow. Hopefully that provides you with the insight on how to fix your issue.

Hi Mike,

There a couple of solutions, neither of them provide you with 100% of data. The best would be to export a list of landing pages from Google Analytics or your favorite web analytics tool segmented by organic search/ Google. This would provide you with a list of pages that received traffic via search and so are indexed. If you cross reference them with your sitemaps that might already help you out a bit. Besides that you could crawl and scrape the URLS for a site:xxx.com search.

Hi,

This might impact your ranking, so you need to make sure if you rewrite the URLs for a page that you redirect the old URL to the new one to make sure you pass all the link juice and users to the new one. This should in most cases make sure that your rankings will remain the same as you pass all the authority to the new ones. If you don't do this than you probably will have to build up your rankings from the ground for these URLs.

Hi boux,

At the moment Google is probably incorporating the CTR in some way to calculate the rankings. They know from the period of time between visiting SERPs how much time you\v'e spend on a page. I however doubt that they will use it to calculate the domain authority. As for some pages it won't say a thing that you've only been to a page for 30 seconds. To be fair I don't think you'll ever be able to calculate the impact of switching off your PPC brand listings to see as positive impact on SEO.