Welcome to the Q&A Forum

Sorry I can't offer any more help here, but you've done exactly all I could recommend to speed up the process - demoting the pesky sitelinks.

Only thing really to add is to ensure that the links that you would want to appear in the sitelinks are recognised by Google as the most important landing pages - so link to them internally wherever you can and of course externally (though of course the 301s will help with this)

Hey there

It's not unreasonable at all that a company may operate from the same premises, both physically and digitally.

In terms of SEO, having 2 sites interlinking with each other that sit on the same IP won't be a problem.  In fact, a few more wouldn't be a problem either.  Matt Cutts recently answered the question on how (and how much) you should link your sites together: http://youtu.be/x0-jw_PfwtY

In short, you can do so happily in moderation - so 2 websites like yours linking together won't be a problem at all.  I'd say you'd need to look at things a bit closer if you have 5 or more.

Hope this helps.

Hi Dario

If you're not intending to restrict access to certain parts of the site to certain crawlers, then you won't actually need a robots.txt file at all.  So unless you have sub-folders that you want to keep private, don't worry about it.

In that case you don't need a robots.txt file at all.

Your example above is actually very close to restricting access to your entire site.  If you had a robots.txt that looked like this:

User-agent: * 
Disallow: /

You would stop any bot or visitor from seeing the site.  Definitely not recommended!

In your case, I would avoid having a robots.txt file altogether.  It's not an SEO requirement until you need to restrict access for whatever reason.

Hi David

Without looking at the backlink profile of either domain I can't be certain, but it very much looks like that 301 redirect has brought about the penalty.

It doesn't matter if there are 30 bad links or 30000, if a Google quality reviewer believes the backlinks are poor quality, you run the risk of being penalised.  I wonder whether you are seeing all of the links at the moment - it might be worth using the Link Detox tool for a more comprehensive backlink audit than Open Site Explorer can offer.  This may reveal more poor quality links.

But it sounds like you're pretty confident that the 301 redirect has caused the penalty.  Rightly or wrongly, I'm afraid it's not up to us to judge what links are "bad", it's Googles.  So while you and I may think the old backlinks are OK, Google may take a different view.  I would also rule out any chance the penalty may have become because of an influx of new links (via the 301) - having set up a number of new websites and redirected old domains (with hundreds of thousands of links) I've never seen this trigger a penalty by itself.

I'd put your theory to the test and remove the 301 and send a reconsideration request detailing you have done so.  From what I can tell, that would remove the penalty.  Run a deeper audit of your backlinks to see if yet-to-be-revealed bad links are present.  And if you really want some authority links from other websites to pass through to the new domain, contact them manually and ask them to update their URLs.

Hope this helps

Hi Mike

It's quite difficult to say, but I would offer this bit of solace: When I've seen sites that have been hit with a Panda penalty in the past, it has happened a day or 2 after receiving a similar warning to the one you have reported.

And similarly, although not the same penalty, people have reported that they have received an unnatural link warning and then 2 or 3 days later have seen the big drop in rankings and traffic.

Both of these instances indicate that the notification of a possibly penalty comes before the penalty itself.  If (and it is an 'if') this is the case with your site, then the big drop in traffic can be attributed to the penalty that you thought was coming, rather than the action you took yourself.

It does sound like the crawl and the warning would result in potential duplicate content issues or low quality content issues, aka a Panda penalty.  I'd take out due diligence and make sure that you have not blocked any key landing pages via robots.txt file or by any other method - always a good idea to double check.  However, if that's not the case, I would say that the drop is because of the penalty that was likely coming, rather than what you've done.

The next step would be to wait until the algorithm refreshes - Panda updates rollout at least twice a month now so hopefully your efforts should be rewarded soon enough and the traffic will return.

However, it is worth considering that just because the Penguin algorithm updates on a given day, does not mean its impacted is limited to the day or the days that follow.  What I think we've seen with Penguin 2.0 is the impact of the algorithm updating has been spread out over refreshes, so we can't completely rule out this as Penguin action as well.  SERP volatility checkers like Mozcast seem to suggest that this probably isn't the case, all was quiet on August 13th on that algorithm, but it's worth bearing in mind.

I hope this helps - do take my advice with a pinch of salt as it's an educated guess from my experience rather than any definitive evidence.

Hi there Ash

On the contrary, I think Google and Bing are recognising the blog and indexing it well.

With this site search, you can see over 89 URLs indexed - it looks as though Google is indexing the blog posts and the categories just fine.

What were you expecting to see from this?  If there was something specific, I can try and see if it's being achieved.

As a couple of general points, it's a good idea to link some of your blog posts to other internal pages on your site, for example linking the words "funeral plans" in your blog posts to your funeral plans page.  That means that if this page gains links/social shares, the SEO "strength" will pass through that blog post to your key landing pages.

As an aside, the URL structure looks pretty long.  With the postID and number subfolders, your blog posts are 5 subfolders deep.  Sometimes this can be a bit of a barrier for Google and Bing to index your page, although as said before that doesn't look to be a problem.  If at all possible, ask your developer to remove those two subfolders - 3 folders deep is usually the recommended site structure.  This isn't that essential, however.

Hi there Sara

GTMetrix will be able to do this for you.  It's a free tool that lets you measure response time and also suggests improvements based on Google Page Speed and YSlow guidelines.

What I'd recommend is registering your email (for free) so you can customise and save your test.  You can tweak it on testing location, what browser to use (specifically for mobile you can select Chrome for Android) and what connection to use - again, you can choose a 3g or 2g connection to get some idea for load times on mobile networks, not just on wireless.

Hope this helps.

Hi Eric

To me, it looks like you've identified the problem. In the technical sense, it looks like your website is suffering from a over-optimisation/Penguin penalty.

If you look at the links pointing to your product pages, there are a number of unnatural signals.  Virtually all of the links come from shademakers.com (unnatural), they all have commercial or targeted anchor text (unnatural) and they are all dofollow, thus passing SEO equity.

The fastest way to remedy this in my eyes would be to remove the links.  You could 301 redirect, but I believe removing the links on shademakers.com looks more like a conscious effort to stop this manipulative linking (as it stops any link equity being passed altogether, which is more what the Google algorithm will want to see).

It's worth noting here that, since this is likely an algorithmic penalty, it may take a while for the removal of these links to be seen and reconsidered by that part of the algorithm (and there's nothing unfortunately you can do to accelerate this).  Similarly, those links were likely to be the cause of the big jump in rankings (in tandem with your on-site SEO), so in order to see top 3 rankings again you may need to earn high quality and relevant links to those pages with healthier anchor texts.  The third scenario is a sort of combination of the two - the bad links have been devalued but no negative action on your site has been taken by the algorithm - in which case you just need to earn high quality links in order to recover the rankings.

I would remove the links completely and try to earn better links.  Once rankings start to pick up, you may want to link from the shademakers.com site again (if it's getting any relevant traffic), but if you do so I would almost certainly use branded or non-keyword rich anchor texts and also use no-follow links - to show that you're not trying to pass PageRank or any SEO link equity, but just want to link to a relevant site.

Hope this helps.

Hi Eric

The manual action appearing in the site is not a bad sign to have at all - it shows that it was not algorithmic action and so now that you've identified the problem you should be able to get it removed ASAP.

Yes, I would remove all the links and then in your reconsideration request mention exactly what you have here.  The manipulative links were unbeknownst to you, you've identified them all, removed them all (and cite the URLs where they have been removed) and you should be good to go.

To me it looks like these were made in genuine error and not meant to manipulate rankings; you should mention that. It's not everyday where they get reconsiderations where 100% of the bad links will be removed, as in your case, so it should be pretty positive.

These requests typically take 5-7 days, but I have seen them take up to 2 weeks, just an FYI.

Hi Iris

I don't think you have anything to worry about here.  This is best practice and having 100 redirect lines in there won't make an impact on site load speed.

It goes without saying that the more lines you add, the larger the file becomes and the larger the file is, the longer it takes to process - but for the .htaccess file I've only ever seen this become an issue when you're looking at 10,000s of redirects.  When you get to that stage, its more of a problem with how you're entering the redirects (using individual URLs rather than redirecting whole subdirectories, for example) rather than it not being good practice.

So, in short, you look to be fine to me!

Hi Tom

(Great name)

If you preference is to have the all.html page (and more.html page) indexed and potentially ranked, rather than the subset/filtered pages, then you should do just as you say.

It's worth double checking if your "more.html" page is being indexed as well (just paste the URL into a Google search bar).  If so, if you can point any of the subset pages to that one, if it is more accurate, then that's an option.  Google, when crawling the page, won't see the dynamically loaded content (at least I think - you can test this by pasting the URL into the SEO Browser, running a 'simple' analysis, and seeing what the Googlebot sees), so it might be wise to point the subset pages to the URL that it matches most closely.

If that isn't an option though, there won't be an issue with adding the canonicals to the subset pages to point to the all.html - even if the content isn't an exact match.  You're taking steps to prevent any duplicate indexation and Google will appreciate that.  Another idea, which is probably more robust, is to add the tag to the code in the HTML, if the listing pages physically exist and are not dynamically created with search queries.

Having said that, I do believe your solution would work fine. Hope this helps

Hi there Jordan

I've just run the page in the Screaming Frog SEO Spider tool and it's found lots of internal inbound links to it.  I can only presume this means it's an error in the Moz tool, so I'll let one of the support team address it.

From where I'm standing, however, it doesn't look like the page has been orphaned, so you're good.

Hi there

John Mueller is right, if you have a canonical tag in place like that then you shouldn't have to worry about those duplicate pages.

The reason why your page variations are being indexed is because you've implemented the canonical tag incorrectly.  At the moment, you have it written like this:

Your missing the quotations from the canonical section.  It should read like this:

That should fix the problem and also fix the duplicate URLs from being indexed.

Hope this helps.

Hi Oscar

I think this could be a very good idea.  Not only does the new URL structure look more SEO friendly for bots and also more friendly for users, creating a category page can be a great way at ranking an ecommerce site.

One of the main problems with an ecommerce site is the requirement for each product page to have its own unique content on it of some substantial length, let's say 200+ words.  If you have hundreds or thousands of products, this quickly becomes a problem.

Conversely, creating a category page that covers your main search terms - eg "sunglasses", "sunglasses for sale"  - can also cover some long-tail keywords as well: "best sunglasses for summer", "sport sunglasses" and so on.

In order to do that you should make your category page as rich as possible - include a healthy amount of content and even things like reviews and videos of some of your top selling products.  Make it informational as well as promoting your shop, so that it could also be seen as an educational resource as well as your shop front.

I think having a static category page over a search result page to promote in organic search is a very good idea. Make sure it is content rich and worthy of ranking and that any necessary redirects are taken care of.

Hi Sagar

If the link profile is bad - which isn't a Panda issue - I would not redirect the domain.  Not sure if there is confusion here in your penalty definitions, so I'll cover all bases.

In terms of a Panda penalty, ie a thin content/duplicate/spammy on-page website, I've yet to see that penalty pass on to a new domain when redirected.  However, a bad link profile penalty (often a Penguin penalty) I have definitely seen passed on through a 301 redirect.  The penalty doesn't take hold immediately on a new domain, but it does eventually and is something of a ticking time bomb.

For that reason, I wouldn't want to 301 redirect the domain until I removed a good majority of the bad links in the link profile of the old domain.

I hope that helps.

While I can't help with the second part of your question just now, I can help with the first.  There have been a couple of studies that have argued that 302 redirects are eventually treated as 301s.  Here is one example.

Hi Howard

Other than using OSE, there are a couple of manual checks I like to take when checking a website's quality.

First of all - how many outbound/external links are there?  By that I mean, how many other sites are they linking to?  Are they linking only to sites related to their industry? How are they linking to them - is it brand name, URL or anchor text?

You don't want to work with a website that looks to be linking to dozens or hundreds of other websites - it might indicate a link farm (plus the link equity/strength that your link would get would be more diluted).  Similarly, if the website is linking to lots of other websites in completely different industries, it may look quite spammy.  And finally, if the website is aggressively using anchor text to link to those sites ("We recommend this company for cheap blue widgets" for example), again it would be one to avoid.

Another check I like to make is the social activity on the site and pages.  Are there facebook likes/tweets?  Comments on articles?  These may not help with your SEO efforts directly, but it's another check to see whether the website is a legitimate website with a legitimate audience, and not spammy or a link farm.  I would not rule out a website on just these grounds, but Google is quite clear in saying they want websites to engage with audiences and earn their links, rather than just get them on "strong SEO" websites.

Hope these two tips help.

I obviously can't say for sure regarding the anchor text - however:

My understanding is that the anchor text ratio is more a signal rather than a trigger - as in it must be one of (X) amount of signals to then fire a penalty trigger (algorithmic/manual review).

So even if it was still sending a bad 'signal' as it were, it wouldn't be the final nail in the coffin.

Of course, this is my opinion, but it's based on my experience and a bit of testing with some spam sites.

Hi Luke

Those hashtags are internal anchors and they are quite commonplace and so Google handles them quite well.

What I would test would be how the search engines are crawling this page?  What should be happening is that the search engine can see all of the text at all times.  You can test this by entering the URL into the SEO Browser Tool - enter it there and click "simple" search (as it's free).  What do you see?  Does all of the page content show up?  If so, Google will be able to see the page.

Now there are a couple of things to consider.  There has been talk that Google will no longer index "click to expand" content.  I've seen conflicting reports as to whether this has been implemented, but if it is on Google's radar it may be wise to avoid the tactic.  From what I can gather, it looks like your page might be acting this way at the moment, in which case you may want to change that.  If your website is using internal anchors (hashtags) to merely take the user from one section of the page to another (while never hiding the content) it would be fine.

Finally, if you're using Google Analytics, by default GA will register a new page visit each time a hashtag link is clicked.  Similarly, it will not show you hashtags have been clicked and show them as visited URLs.  So, for example, if someone visits www.example.com/page and clicks 3 hashtag links, that will register as 4 page views to www.example.com/page and nothing else.

However, you change that and here's a good guide on how to do so.

Hope this helps.

To add to Matt's good earlier response, the TLD extension that you use (.com, .net, .info) will have a minimal impact on your ability to rank well and the site's overall SEO.  For ranking specific websites in specific countries (and regions like London etc), Google looks at a bunch of more important factors, such as server location, schema, hreflang setup, citations, NAPs and so on.

Hi Pete

Sorry that I missed your comment before.

I can absolutely understand why there would be a sense of frustration for designers and webmasters here.  First of all, as Patrick alludes, it is not exactly clear whether or not this is being accounted for (results and feedback are inconsistent, to the point where I can't see a pattern in industries/niches etc).  And secondly, Google has always been about making the page as friendly for the user as possible.  Using expandable content/internal anchors can do just that, so it's a frustrating contradiction.

My advice would be what Patrick has said and that's to test.  I'd give it 6-8 weeks to try and see any substantial change.

Hi Amelia

Would a subfolder not achieve the job equally as well?  If we take out the SEO perspective (I think any links the forum eventually gets you would have virtually the same benefit in a sub domain or sub folder), let's look at it from a user POV.

As you say, your audience responds very well to forums.  How do you think those users usually find them?  Would they be more likely to type in www.example.com/forum/ or would they type: forum.example.com?  My gut feeling (based on no data) would be the subfolder solution - as it's the same way you might go about finding a contact page or an about us page.

You could attempt to quantify that by doing a google search.  Try this Google search:

inurl:forum "keyword"

Replace keyword with your industry keywords, of course.

Take a look at some of the results that come up and note which ones use a /forum/ structure or a forum.site.com structure.  Both variants have the chance of turning up (like in this search).  That will give you some idea on what the typical structure is.  Now, that doesn't mean you have to be typical!  But it might give a slight correlation between what structure users in your niche might expect.

Ultimately, I don't think it would make a huge amount of difference - so maybe giving the devs an easy ride could be a good shout!

Hi Nikola

It certainly won't be the case that, if you have a big rankings drop, that might have a ripple effect on other rankings.  There's no way the two will be directly related and Google won't adjust your other rankings because of one falling.

However, the reason behind the ranking drop might well have contributed to other keywords falling, particularly if they are related keywords.  So, if a competitor has had a successful link building campaign and risen above you, you yourself have lost links, made onsite changes etc etc.  Those ranking factors can all increase or decrease your keyword positioning - and if all of your keywords are related and possibly ranking for the same page, that might explain why other keywords have fallen as well.

Hi Sarmad

On the whole, the health niche is very competitive.  You may have found a nice little pocket to target, but the likelihood is that you'll be going up against some big brands and operations, so whoever you choose to work with should be obliged to give you an frank estimation - which may involve a budget bigger than expected.

Be wary of those PMing you promising the world at a low cost.  If it sounds too good to be true - it probably is.