Welcome to the Q&A Forum

...... it depends!

It depends on the industry you are targeting, the quality of your link requests (personalised, relevant etc.), how many sites you are targeting and their status in the industry, what the other players are doing and probably lots of other things...

I would be really hesitant to put a number on it, because it is depending on all these things. However, if you are writing personalised requests to relevant sites, and are offering some value to those you are requesting links from, then I'd say 1 in every 2 or 3 is a good hit rate to aim for initially if you want a target.

Great question!

Firstly - unfortunately, Steve's suggestion isn't going to be viable for you. The # portion of the URL is not available to your code server-side, so you won't be able to determine where the rel canonical should point.

Furthermore, if they are committed to keeping the flash for now, and all as a single unit so one URL (the homepage), then you are going to have to accept that some juice intended for subpages is going to go to the homepage. You cannot do anything about that aspect, so you need to focus on the rest of the problem. However, whilst far from ideal, at least the juice is hitting the site somehow.

So… what to do?

Firstly, I'd start getting into the mindset of thinking in terms of the HTML site as the main/canonical site, and the Flash site as the 'enhanced experience' version. In this way, the HTML version is going to be the version that should be crawled by Google, and should be linked to.

Actions:

1. Setup detection for mobile user-agents (out of preference I'd say all, but at least those known not to support flash, such as iPhone/iPad) and search engine bots, and ensure they get served the HTML version. Currently your homepage requires a click through on iPad offering an impossible Flash download, why not serve them the HTML page off the bat.

Is this cloaking? No! The HTML version is the main version, remember? It's no more cloaking than if you detected the user agent and then chose to serve the Flash version to Googlebot.

I actually discussed this with Jane Copeland at the fantastic Distilled link building event a couple of weeks back, and she agreed with me and said if it would stand up to a manual inspection then it is the right course of action.

2. Get all links in articles, press releases, directories or whatever else that are linking to specific pages and are originating from in house (or any source you have control over) to link to the HTML pages.

3. If the user arrives, has Flash and has arrived to an HTML link, you can now redirect to the Flash link for that page so they get the 'enhanced experience'. Don't use a 301 redirect -- remember the HTML version is the main version!

4. If the user arrives via a Flash link, but doesn't have Flash, but does have javascript you can detect the # variable and redirect them to the HTML page to help them along.

5. Educate the relevant stakeholders regarding point 2. I see you have a 'flashmode=0' option, tell them about this and how to use it get the URLs they need.

So where does this leave us?

1. The search engines can crawl all your lovely content, and they can ignore the flash version completely.

2. You are getting inbound links to specific pages. These pages have their own titles and meta descriptions… and content! Because they are the real site!

3. Users with Flash arriving via these links are landing on the correct Flash page of the site and are experiencing the rich site that you want them to.

4. Users arriving without Flash are getting the correct page if they arrive via an HTML URL. If they arrive via a Flash url then they get the correct page if they have javascript on (e.g iPad users), or they get the fallback of the homepage (rare).

I had a client with an almost identical situation, and I rolled out an almost identical solution to this, and they got crawled very quickly, shot up in Google and have stayed there for months.

Hope it helps. Let us know how you get on!

Hi Asif!

Firstly - I'd suggest that as soon as possible you address the core problem - the use of session ids in the URL. There are not many upsides to the approach and there are many downsides.That it doesn't show up with the site: command doesn't mean it isn't having a negative impact.

In the meantime, you should add a rel=canonical tag to all the offending pages pointing to the URL without the session id. Secondly, you could use robots.txt to block the SEOmoz bot from crawling pages with session ids, but it may affect the bots ability to crawl the site if all the links it is presented with are with session ids - which takes us back around to fixing the core problem.

Hope this helps a little!

I'm not sure if anyone has any solid data on this, and even if they did I think it would be so conditioned on many other factors that it wouldn't be much help.

The pro's of allowing dofollow links from commenters on your blog is obviously that it encourages people to get involved, and maybe they stick around and become part of the community. The downside is the spammers who will invade!

You can argue both sides...

If people are going to bother to comment with good quality comments which are useful to building a community, and for they themselves to be persuaded to stick around is going to require that you are creating great content. If the content is crappy then people won't bother crafting great replies. So if the quality is crappy and you allow dofollow then you'll get (other than the spammers...) people who are maybe nice but aren't really interested in your community, they just want a link.

if the content is great then do you need the dofollow to persuade people to comment?

On the flipside, if the content is pretty good maybe a dofollow link helps to push people over the edge and make them bother to write the comment... If the community is small you should be able to manage the spam.

At the end of the day - I think you need to test in your niche and see what happens... Good luck!

Hi!

There shouldn't really be a necessity to mix rel=canonical with meta noindex; the rel=canonical would take care of it without sending mixed signals to Google.

For your scenario, how to treat the paginated pages depends on whether they are the only/best way to discover the products listed on those pages. If they are, then noindex, follow probably makes sense. I think it would likely be the best answer.

Good luck!

I would say a bit of both. It is fine to repeat your primary keyword phrase several times on the page; the number of times depends upon the amount of content. SEOmoz's On Page tool recommends 4 repetitions. However, you should also try to use some synonyms and secondary target keyword phrases also.

A good resource I saw posted today which might be of interest:

http://lsikeywords.com/

If I have understood it correctly, you want:

• To redirect certain category pages to new category pages on a new domain.
• Redirect all other pages to the homepage on a new domain
• Not to redirect the homepage and for that to stay on the current domain.

If so I think this should probably do what you want:

RewriteRule ^$ - [L]
RewriteRule categoryA/ http://www.newdomain.com/newcategoryA/ [R=301,L]
RewriteRule (.+) http://www.newdomain.com/ [R=301,L]

The first rule says "if this is the homepage stop evaluating rules". The second maps a category page over to the new version and stops evaluating rules; you'll need to duplicate this one for all the specific categories. The last rule is your catch-all rule for redirecting everything else to the homepage of the new domain.

If you want to easily test/check then I highly recommend this .htaccess checker tool.

Good luck!

Hi Stephon,

At $99, the monthly cost of the SEOmoz pro membership would be insanely cheap if it actually performed automated SEO.

However, if you are running a business that benefits from an easily discoverable website (almost any business!), and you've not yet performed any SEO then there is almost certainly going to be a very high ROI on doing so.

If your budget doesn't reach to hiring an SEO company, then you could consider learning the basics yourself. The book "The Art of SEO", combined with the tools on offer at SEOmoz could get you doing the basics yourself quite quickly, and if you stuck at it you could move on to the advanced stuff.

Otherwise, there are many great SEO companies out there, but be sure to do thorough homework before you go for it with anyone.

Best of luck!

Hi there!

Ok, it is difficult to know all the ins and outs without looking at the site, but the immediate issue is that your robots.txt setup is incorrect. robots.txt files should be one per subdomain, and cannot exist inside sub-folders:

A **robots.txt **file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers

From Google's page here: https://support.google.com/webmasters/answer/6062608?hl=en

You shouldn't be blocking Google from either site, and attempting to do so may be the problem with why your hreflang directives are not being detected. You should move to having a single robots.txt file located at https://www.clientname.com/robots.txt, with a link to a single sitemap index file. That sitemap index file should then link to each of your two UK & US sitemap files.

You should ensure you have hreflang directives for every page. Hopefully after these changes you will see things start to get better. Good luck!

Ok, cool. To reiterate - with escaped_fragment you are just serving the same content in a tweaked format and Google recommend it rather than frown upon it. Good to be sure though.

See you at SearchLove!

Putting aside server load / config issues, and from the pure SEO point of view.

No, you shouldn't have any major issues with that many 301s. However, what you might find is that depending on the size of your site and the frequency of Googlebots visits that some of these pages take a long time (months) to drop out of the index and be replaced by their newer alternatives. This normally isn't cause for alarm.

In some instances you might end up with pages that now have now links to them (as their parent categories were all redirected also) and so seem to get stuck and never get recrawled by Google to update. In a couple of instances I have had success using XML sitemap files that just include these 'blocked' pages (the old URLs still in the index) to prompt Google to recrawl them.

Also there is Google Webmaster Tools feature to 'crawl as Googlebot' which then prompts you to 'submit to index' which you can use to prompt recrawls on a per-page basis (but you have credits here, so should only be for the more important pages).

Best of luck!

Here you go, I managed to dig up informative blog posts for most of the active tools:

Trifecta

http://www.seomoz.org/blog/goodbye-page-strength-hello-trifecta-seomozs-latest-analysis-tool

Competitive Link Finder

http://www.seomoz.org/ugc/competitive-link-finder-provides-qualitative-link-building-ideas

Linkscape

http://www.seomoz.org/blog/linkscape-education-series-learn-the-ins-amp-outs-of-how-linkscape-works

Juicy Link Finder

http://www.seomoz.org/blog/new-premium-services-link-directory-juicy-link-finder-tool-new-pricing

Crawl Test

http://www.seomoz.org/blog/crawl-test-seo-tool

Blogscape

http://www.seomoz.org/blog/announcing-blogscape-and-seomoz-labs

If I'm reading this right though, it is only the URLs they've got to stop using, not the content. Therefore a 404 provide alternate content suggestions isn't necessary in this case; I agree that a 301 redirect is best solution - it passes the human traffic and the link juice to the correct location.

As to whether it is worth the cost, then of course it is the famous answer of "it depends". However, I'd imagine that the cost of redirects should be pretty minimal and if the old URLs drive just a couple of conversions (whatever that may be) then it should have been worthwhile, even ignoring the link juice.

Just to chime in on this, albeit maybe a little late now... I had the same thought as I was reading through this with using rel=canonical to point the old pages to the new for now, so the search engines don't have any duplicate content issues until a 301 redirect can take over when the new site is fully launched.

However, depending on your rollout schedule, this would mean that the SERPs would soon be indexing only the new pages. You'd need to ensure that the traffic diverter you are using would handle this. Otherwise you could put the rel=canonical on the new pages for now, which would avoid the duplicate content until you are fully launched. Then you'd remove it and 301 redirect the old pages to the new.

Just something you maybe want to think about! Hopefully your traffic diverter can handle this though.

Hey there,

Unless I'm missing some information none of the rules there would redirect either of those pages to the homepage. Is there more of the file?

-Tom

Hi Jonathon,

Rand mentioned at the amazing Distilled Link Building event last week that this was something that is coming in SEOmoz Pro tools. Hang tight, and I think we'll have it soon.

Hi Ben,

The benefits include:

• having a shorter URL

• moving keywords to earlier positions in the URL

• making the pages appear not so deep in the site structure

Some people have said they don't want the date in the URL to devalue a post in a readers mind, where others have said it is useful information for a reader to know the date of a post.

As with most things SEO, it is a case of 'it depends'.

I would add that the 301 redirects shouldn't take too long to sort out and test, and so don't let that weigh too heavily into your decision; decide what you think is the best decision from the SEO point of view and then worry about if that means you need to do 301 redirects,

-Tom

I will just add that you need to be very careful. Maybe try the ones out that everyone has kindly suggested and then use SEOmoz Campaigns or other tools to check the quality of it.

This Whiteboard Friday post from last week has one of Bing's team pointing out how it is very important that your sitemap is of a high quality (no 404s, 302s or 301s for example), or it could be ignored completely:

http://www.seomoz.org/blog/bings-duane-forrester-on-webmaster-tools-metrics-and-sitemap-quality-thresholds

If you have many URLs from the old site in the index that are all in the same directory (or a handful of directories) you can quickly and easily remove whole directories of URLs from the index via Google Search Console. We have found it to work very quickly.

1. Go into Search Console, selected ‘Remove URLs’ under ‘Google Index’ in the left hand menu.

2. Add the page or folder you want to remove, and click next. If you add the homepage, that's the same as all pages on the site. If you add a folder you'll get three options under the ‘Reason’ drop down.

One of those options is ‘Remove directory’. Select that.

Hi Mark,

I've taken a look at your domain in both OSE and Majestic, and don't see anything untoward. I've checked all your 301s and it looks all configured correctly to me.

I see a very very slight difference in the sub-domain and root-domain numbers, which isn't unusual and which I see quite often.

I don't think you have anything to worry about!

Hallo Bernd,

Of course, I agree with everyone else that you need to fix your robots.txt file.

However I'd also add the suggestion that you setup Google Webmaster Tools for your site. These will help inform you about crawl errors and your robots.txt file and might be helpful for you in future.

Also whilst having a quick look at your site I noticed some duplicate page title issues. Make sure you are tracking your site with SEOmoz's campaign tool. It will really help you find these types of issues.

Viel Glück!

Hi David,

Google/Bing etc. have very few problems recognising such characters in the Latin alphabet. It looks like you are mainly concerned with umlauts, which Google handles intelligently. For example...

• Google will identify the difference between a search for "Küchen" (kitchens in German) and "Kuchen" (cake in German) and offer up relevant results. This is true in Google US and Google UK, not just localised Googles.

• Search suggestions work just fine with these characters, and even with the standardised way of rewriting them when there is no accessible way to type them (for umlauts this is with an e following the letter). For example, in Google.de, type "Kue" and you will be given the suggestion "Küchen".

You are mainly concerned with brands, which muddies the waters a little because many people in English speaking markets won't bother/know how to type the umlauts. However, Google normally handles this well and recognises the intent.

I would recommend you ensure you consistent use the brand name with the foreign characters, as intended. Google/Bing and co. shouldn't have any problems. Which HTML encoding you use is by the by, in my opinion, as long as the characters are rendering correctly.

From a crawler point of view, I don't believe there is any difference. However, MaryAnneG makes a good point about scrapers. The downside of absolute URLs is they are harder to maintain, especially if you move to a new domain or change the structure of your site.

Hi Philip,

I am not sure I understand this part of your question "does the rating system have to be present and possible to rate?".

You don't need to have anything on the page that enables someone to leave a rating/review in order to use structured markup on those ratings/reviews.

I would recommend using an approach combined with rel-alternate-hreflang to link the various product pages in different languages together.

My approach would be to give the star rating (marked up with http://www.schema.org/AggregateRating) gathered across all reviews for that product (in any language). Then on each different language page I'd show the reviews that are in that language (http://www.schema.org/Review).

I have never handled this exact situation, but that feels like the right way to approach it. What do others think?

-Tom

Hi August,

You've made a small error; it looks like you've used the rel canonical example from here; http://www.seomoz.org/learn-seo/duplicate-content. The "canonical-version-of-page" there is supposed to be an example of a folder on the site and shouldn't be literal, so the code on your page should be:

Where the value inside the 'href' part is the URL to the page you wish to be the canonical version. I hope that makes sense!

What Francisco is suggesting is an alternative, and often preferred, method of handling this scenario, where a user trying to visit the other (non-canonical) versions of this URL would be redirected automatically by their browser to the canonical version. This does have some advantages but I'd say it isn't significant enough for you to worry about.

Best of luck!

I would differ slightly in my approach. If you robots.txt block the other sites, then any organic links they build up will be worthless.

Rel canonical across domains should be fine, so put that in place. Then meta noindex, follow: this way the juice flows in at least. Make sure the rel=canonicals go to the same specific page on the duplicate, obviously.

Well, it depends! Use of the rel=canonical tag sounds fine to me, I agree they'll feel more 'at home' with a .de domain.

However, what is the strategy for the German users to find the .de version of the site? I'm guessing AdWords or similar where you display the .de domain to increase the CTR. However, it would be nice to be able to bring users in via the organic rankings.

Seeing as this is a valid problem faced by many, it is a shame the engines haven't yet provided a better way to handle it.

Viel Glück!

Yep, I think I agree with your point about new links = quick boost, and old links = authority. This agrees with my general view that old links don't weaken with time, and also form a solid base for your link profile. However, hearing these conflicting viewpoints recently really piqued my interest.

Hi Panos,

I don't necessarily disagree with Eric's answer, but I wanted to answer from a different point of view. I'm going to assume you really want or need some refresh mechanism built into the page.

In which case I'd agree that a Javascript approach using AJAX is probably a better solution. It will mean that users only need to load the new article headlines, and not the whole page, so the strain on your servers should be reduced. Furthermore, I find it a neater solution all around anyway - you could provide a notice 'new headlines available' that people click to refresh the articles list. This might be the best of both worlds?

Either way, meta refresh isn't as flexible, isn't as clean, and will put more strain on your servers.

Good luck!

-Tom

So, there are lots of 'ifs' here, but the primary problem I see with your plan is that the CDN will return the content to Googlebot without the request hitting your server so you won't have the option to serve different headers to Googlebot.

Remember that every page is the main HTML content (which may be static or dynamically generated for every request), and then a whole bunch of other resources (Javascript and CSS files, images, font files etc.). These other resources are typically static and lend themselves far better to being cached.

Are your pages static or dynamic? If they are dynamic then you are possibly not benefitting from them being cached anyway, so you could use the 'vary' header on just these pages, and not on any static resources. This would ensure your static resources are cached by your CDN and give you a lot of the benefit of the CDN, and only the dynamic HTML content is served directly from the server.

If most of your pages are static you could still use this approach, but just without the full benefit of the CDN, which sucks.

Some of the CDNs are already working on this (see http://www.computerworld.com/s/article/9225343/Akamai_eyes_acceleration_boost_for_mobile_content and http://orcaman.blogspot.co.uk/2013/08/cdn-caching-problems-vary-user-agent.html) to try and find better solutions.

I hope some of this helps!

Hi,

What happens when using HTTP? Are you able to share the .htaccess code?

The same .htaccess setup should work for for both protocols, but with .htaccess the devil can be in the details sometimes. A good tool for testing it is here: https://htaccess.madewithlove.be/

Do other redirects from HTTP to HTTPS work ok? I'm wondering whether your Apache setup perhaps has a different virtual host setup for HTTP and thus a different .htaccess file.

Best,

Tom

For the amount of effort you spend on such a scheme, you could do a lot better by just trying to think outside the box, and thinking more creatively than the others in your niche. When you are forced to think of a solution that is purely white hat that can achieve or outperform a grey/black approach, you often come up with a solution that whilst maybe not so quick is going to be more sustainable and add value to your users.

I encourage you to think about what would be really awesome content/tools/linkbait for your industry, and what hasn't been done or done well (or not been updated for sometime - this is a good one), etc etc. Then do that. You will not be disappointed you bothered.

Hey Dirk,

No worries - I visited the question first time today and considered it unanswered as the site is perfectly accessible in California. I like to confirm what Search Console says as that is 'straight from the horses mouth'.

Thanks for confirming that the IP redirect has changed, that is interesting. It is impossible for us to know when that happened - I would have expected thing to get indexed quite fast when it changed.

With the extra info I'm happy to mark this as answered, but would be good to hear from the OP.

Best,

-Tom

I think your best bet is rel=canonical. Furthermore, Google agree:

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=139394

This section is directly related to your issue:

How do I specify a canonical page?

snip

If you publish content on both http://www.example.com/product.php?item=swedish-fish and https://www.example.com/product.php?item=swedish-fish, you can specify the canonical version of the page. Create the element:

Hope that helps.

Hi,

I am not sure I follow your concerns around serving an alternative version of the page to search engines - is that concern based on concerns it will be frowned upon or technical concerns?

Using the escaped_fragment methodology would work for your purposes, and would be the best approach. If you have technical concerns around creating the HTML snapshots you could look at a service such as https://prerender.io/ which helps manage this process.

If that doesn't answer your question, please give more information so we can understand more specifically where you concerns are.

I like your second screenshot with the links to the recent issues that persist over all the pages, and think keeping them separated by year makes them easy to navigate for users and easy to crawl for search engines.

I would comment, whilst you are restructuring, that your content is very deep. It seems everything on your site exists at least 2 folders deep:

/online/en/home.html

Given your site seems to be only available in English I don't see the necessity of the 'online' or the 'en' directories.

Secondly, the publications:

/online/en/home/publications/journals/afp/afpsearch.html

Here having 'publications' and 'journals' seems superfluous, and now a 'home' directory has also appeared. Your content is now 6-7 levels down! I'm unsure of what CMS / software is powering the site, but I'm sure this could be changed. So you could have much cleaner, which is much easier:

/journals/afp/search.html

instead of:

/online/en/home/publications/journals/afp/afpsearch.html

Which I think would be much nicer for users and engines.

Best of luck!

Hey,

As Jane mentioned - .htaccess can't change the HTML content of your pages, so you can't replace all the links that would be seen by a user or a browser.

If the links are via a redirection page on your site then you can use .htaccess to redirect those requests elsewhere, so when a user clicks a link it takes them somewhere else instead, but only if those pages are on your site.

Make sense?

Good luck!

-Tom

Hi,

1. I have used CloudFlare for a few sites and never had an issue with this. It is a risk/concern with all shared hosting, but CloudFlare are very proactive about addressing anything impacting their customers, so I would not have a concern on this side of things at all.

2. Again, I wouldn't have concerns here. CloudFlare are very adept at handling large-scale DDOS attacks . Having read some of their post-attack analysis reports, they usually mitigate any impact to customers very quickly. They have loads of customers, and if this sort of thing was an issue I think we'd hear about it fairly often.

3. I can't speak to the % of users that might get falsely identified as a risk and presented a CAPTCHA, but I'd be very surprised if it was as high as 1-2%; I've rarely seen that CAPTCHA screen myself. You should check what CloudFlare have to say on this issue, but I would have no concern here either.

I have never had an issue with CloudFlare impacting SEO performance or impacting the user experience. It has generally performed well for me, but the biggest issue I see with it is people hoping it is a 'cure all' and means they don't need to properly address issues affecting the performance of their site. If your database performance is very poor, meaning dynamic pages take a long time to load, then CloudFlare is not the answer (it may help - but you should address the underlying issue).

I am unsure about the issue with CloudFlare failing when your server is slow - I'd imagine CloudFlare support could help you with this - there may be a configuration option somewhere.

Overall - my suggestion would be that you go for it.

Hi,

So it sounds like you have ~20 internal links per page, which is not unreasonable and shouldn't be a point of concern.

However, if you are still worried, then I'd worry less about finding a specific list of links to dig into - at this scale that likely won't be very helpful or actionable.

I'd recommend that you reviewed the "Top Pages" report in OSE to see which pages have a large bulk of links. Then I'd do a review of the internal site architecture looking at the major elements (top and side navigation, footer links, any dynamic menu systems) and see whether there are places just to tighten that up a little. You can use what you learnt from the 'top pages' report to help inform this review.

However, as I said - I'd certainly not feel you need to go overboard unless you have additional reasons to think that this might be a problem.

Good luck!

seoelevated is correct.

Reference example from Google here: https://support.google.com/webmasters/answer/189077?hl=en

So Google just announced the +1 button, which is there answer to Facebook 'likes' button:

http://searchengineland.com/meet-1-googles-answer-to-the-facebook-like-button-70569

http://techcrunch.com/2011/03/30/google-plus-one/

They should begin showing up now on Google US. If you aren't seeing them, you can turn them on here:

http://www.google.com/experimental/

I want to know? What affect do you predict these are going to have? Is there anything we can do from an SEO point to begin leveraging +1 right from the outset?

There are 2 aspects to consider when you discuss gaming it. The first is the effect it has on the SERPs for everybody, the second is for how it affects it for social users and what the effect on CTR is.

For the 1st, the answer is - the same way they do with links. Check for cliques, for lots of unused accounts +1'ining the same sites, etc. etc. We'll see how effective that is.

For the 2nd, one important fact is going to be you can't game people's networks. You can create a bunch of bogus accounts, and create +1 groups for circular +1'ing. However, none of that matters if you aren't in people's networks. Currently it seems that +1's only show up if at least one friend/contact has +1'd the site. Then you see the total count (see attached image), otherwise at it stands you aren't going to affect people's CTR (which itself will be a signal).

5wu4O