Welcome to the Q&A Forum

Hi Anirban,

Your code is absolutely fine and nothing wrong with it. Just adding the required code does not guarantee that Google will start showing the sitelinks search box for your website. Here is what Google has to say in this regard:

"Search box not displaying? The Sitelinks search box appears only for navigational queries and when relevant for users. Google algorithms use a variety of factors to determine when the search box appears, including the information on the site and different types of navigational queries from Search users."

While there is nothing to fret about, looking at the authoritativeness of the website in question, you should start seeing a sitelinks search box depending on the type of queries users use in Google. When Google finds your website eligible for a sitelinks search box, you will most likely get a message to your webmaster tools account. Chances are more for a website that received very high number of branded queries.

Here is a Moz article in this regard: https://moz.com/blog/sitelinks-search-box

Hope it helps my friend.

Best regards,

Devanur Rafi

Hi,

You should avoid URL canonicalization chains just like you avoid redirect chains. If URL1, URL2, URL3 contain substantially similar or identical content and if you choose URL1 to be the canonical/preferred one then here is what you should ideally be doing:

URL 2 --> Canonicalized to URL1

URL 3 --> Canonicalized to URL1 and not to URL2

In this case, only URL1 will be in Google's index. Here you go for more:

http://moz.com/blog/rel-confused-answers-to-your-rel-canonical-questions

Check out point 9.

Best regards,

Devanur Rafi

Hi,

Here is my view:

All the websites no matter whom they belong to, should have substantially unique content. We need to keep in mind about the very purpose behind why Google exists and what does it take to be able find a place for your website in their index. If two pages serve identical content, why should a search engine like Google index both?

A good example would be, affiliate marketing setup where the affiliate website contains exactly the same content that the original product website. Even the affiliate websites should have their own unique version of content. They should also take up the effort to re-write the product descriptions so that they are not exact copies of the ones found on the original product website.

Coming to your case, by making the content on all the consultants' websites unique, they stand a better chance to rank for corresponding keywords/phrases in search engines. If I were you, I would have re-written the content so that it becomes unique. Hope that helps my friend.

Best regards,

Devanur Rafi

Hi,

Making the content being loaded by AJAX crawlable by Google involves serving a static HTML snapshot of the content being loaded by AJAX to Google. We should make sure that the HTML snapshot is the exact copy that will be served to the visitors through AJAX.

Here you go for more information:

https://support.google.com/webmasters/answer/174992?hl=en

Best regards,

Devanur Rafi

Hi Pete,

Please don't block access to foreigners. You can set the geo-target in Google webmaster tools account, if you have not already done so. It is completely normal with country-specific websites getting traffic from all over the World. A small or probably 1 in a thousand foreign visits might come from UK people traveling abroad. So to conclude, its my honest opinion not to block access to foreigners even the bounce rates might be alarming. Its common and natural and search engine like Google know about this pretty well and there won't be any issue to your website from their end.

Best regards,

Devanur Rafi

Hi friend,

Here is my take on this..

I would tag all the affiliate links with the 'nofollow' attribute no matter what the consequences might be as in the long run it would place you in the good books of Google as its a good gesture as a webmaster.

Here is an article from SEL in this regard:

http://searchengineland.com/googles-matt-cutts-on-affiliate-links-we-handle-majority-of-them-125859

Best regards,

Devanur Rafi

Adding to the above responses. Just because of the fact that Google does not use the keywords meta tag for ranking purposes at this point in time, populating it with too many and irrelevant keywords/phrases can do more harm as doing so is nothing but abusing the tag and moreover it clearly conveys the intention of the webmaster which is none other than to manipulate the search results and there by to rank high for those keywords/phrases.

So you better leave this tag empty if your CMS generates the tag automatically.

Best regards,

Devanur Rafi

Hi friend,

Before getting to the issue, I would like you to please head to Google, type, youtube in the search box and hit Enter. When you see the sitelinks search box for Youtube, type anything in it and hit Enter. You will see list of items from Youtube in the form of an SERP and it will not take you to Youtube directly. This is how sitelinks search box works and the sitelinks search box will not take the visitor directly to the website in question.

Another example would be, to type hayesandjarvis in Google and do a search in the sitelinks search box.

Please let me know if I am missing anything here. Please feel free to write back.

Best regards,

Devanur Rafi

Adding to other responses...

http://googlewebmastercentral.blogspot.com/

And

http://insidesearch.blogspot.com/

These give invaluable information right from the Horse's mouth..Google

Best regards,

Devanur Rafi

Hi,

Content scraping is a very common thing and fortunately, with the rel=canonical tag still pointing to your domain, there is nothing to worry from content duplication point of view and with this tag intact, their content will not make it  to Google's index. You don't need to worry about the other website out-ranking you and you don't have to use disavow here as it is for spammy backlinks and not for rel=canonical tags. However, you can approach those guys and ask them to take it down immediately and you can file a DRM or DMCA case against them if they don't agree to take the content down. You can also report the matter to Google here:

https://www.google.com/webmasters/tools/spamreport?hl=en

Hope it helps.

Best regards,

Devanur Rafi

Hi friend, IMHO, you should go ahead with the merger as the idea is to have one stable and wide website. Moreover, all the SEO goodies of other websites will be inherited by the destination site within few weeks. Don't worry about the product site outperforming the main site. We did this  twice in the past and came out with flying colors. Go ahead with a proper page-to-page 301 redirection for the merger and don't kill the product site for at least 6-7 months.

Please write back with more queries.

Best regards,

Devanur Rafi

I agree with Dennis. An on-going content strategy to add new keyword/phrase specific pages to your domain not only increases the chances of your website ranking high for those but will also make your website look more wide and stable going forward. This way, you could have a substantial growth in your organic traffic in the days to come.

Best,

Devanur Rafi

Hi,

Websites or web properties like Wikipedia do not need a single backlink to rank at the top position for keyword or a phrase. Its the quality of their content that plays a crucial role along with a huge domain authority. Of course, it also matters how competitive the keyword in question is.

By the way, no link profile analysis tool or all those tools put together on this planet can give you the complete list of backlinks and there might be many backlinks that Google would have found and considered for ranking that were not reported by any of the other link profile analysis tools out there.

Best,

Devanur Rafi

Assuming the website in question is meracareerguide.com. Check out this page for example:

http://www.meracareerguide.com/hiii-i-am-a-student-of--12-class-studying-pcm-i-am-confused-whether-to-go-for-iits-or-to-go-for-indi

Here is what's found on the page:

Hi ronit,

The college selections depends on the course you are interested in. ISI offers two bachelors courses B.statistics and B.maths.
Where IIT's are the best choice to opt for Btech courses. Would suggest you to decide the courses of your interest and choose college according to that.

All the best

The content is not only sub-optimal from Grammar point of view but also from the English language structure perspective. Such low quality content does not tend to rank high in search engines. You should get the content thoroughly proof read before posting it on your website. This is just one such an example.

You should also revisit your website monetization methods as the pages are dominated with ads than quality content. For example, this page:

http://www.meracareerguide.com/questions/i-am-in-class-12th-studying-pcm-and-confused-what-should-i-do-after-12th-where-i-should-apply-and-ev

You should seriously look at your URL structure also. There are too many issues with the website from an SEO standpoint and only a qualified SEO expert can guide you in the right direction.

The content should be unique, accurate, up-to-date, very useful to your visitors and above all, should serve the purpose of their visit to your pages.

Unless we have a look at the website in question, we would be just throwing stones in the dark.

I would highly recommend you to get your website audited by a qualified SEO expert.

No matter which wave of Panda update from Google, if your website has been hit by it, it clearly tells you to have a look at your content and its quality. Thin content pages and pages with low quality content will be the victims of Panda. This is the time revisit your content strategy.

Now, this is a very open-ended question and unless we do some SEO forensics or an audit, we might not be able to give specific reasons for the drop in traffic. If you can share the details of the website, probably, we will be in a better position to address your query.

Best regards,

Devanur Rafi

Oh yes, its a paid tool and the free version will not give you the whole picture. Thought it would be helpful..

You are welcome my friend. You can try this one also:

https://fruition.net/google-penalty-checker-tool/

Best regards

Hi,

You can try the following tool:

Llogin to your Google Analytics account and visit this URL:

http://www.barracuda-digital.co.uk/panguin-tool/

Give read-only access to the application and check if this website has been hit by any of the Google updates like Panda or Penguin. That's a quick way to see if you were hit by a Google penalty.

Best regards,

Devanur Rafi

Hi,

If this proxy website is a clone of the original website and if its not blocked thoroughly from being accessed and found by the search engines, we have a huge problem and this proxy website should be taken down ASAP as this might create issues because of duplicate content especially if the original website is fairly new and the proxy website has some strong SEO factors like domain age, domain authority or page authority etc. Please do write back in case you would like to give some more details about the issue or have queries in this regard.

Best regards,

Devanur Rafi

Hi Benny,

Here is a link removal flowchart from BruceClay that can be handy while you audit the link profile of a web property:

http://www.bruceclay.com/blog/link-removal-flowchart/

Hope it helps.

Best regards,

Devanur Rafi

Oh ok...

This line ensures its a 301:

RewriteRule ^(.*)$ https://%1/$1 [R=301,L]

You can use any of the HTTP header checkers available online for the purpose or a desktop tool like Screaming Frog SEO Spider can also give you the HTTP header status codes for any URL.

Best regards,

Devanur Rafi

Oh my God Fred!! the weekend sub-domain has been completely blocked from being crawled using a robots.txt file sitting in the root of the sub-domain.

http://weekend.visitsweden.com/robots.txt

User-agent: *
Disallow: /

Please remove the  '/' from there **immediately**

Hi Fred,

I just copied my first response:

Here is your redirection setup:

http://weekend.visitsweden.com/      being redirected via 301 to

http://weekend.visitsweden.com/no    being redirected via 301 to

http://weekend.visitsweden.com/no/

So, I would suggest you to remove the interim URL without the trailing slash after 'no'. Let the original homepage, http://weekend.visitsweden.com/ also be redirected to http://weekend.visitsweden.com/no/ (the one with trailing slash) via 301.

So your redirection setup should be as follows:

http://weekend.visitsweden.com/  - via 301 to - http://weekend.visitsweden.com/no/

Essentially, we are eliminating the redirection loop here. Please try this and post back.

Best regards,

Devanur Rafi

Hi, absolutely yes, you should go with /wordpress-live-chat as the URL clearly conveys the purpose of the page to both the search engines as well as the users.

Best regards,

Devanur Rafi

Hi, the 404 errors are pretty bad and en user experience standpoint and so Google does not like them. During domain migrations, the most important aspect is to control the number of 404 errors to the possible extent if not possible to make them zero.

When pages are moved, you should go in for a one-to-one or page-to-page server-side, 301 permanent redirection from the old pages to the corresponding new locations so that the old pages do not end-up in 404 errors and with 301 in place, Google will know that the old pages are no more in force and that they have been replaced by the new corresponding destinations. This will also make the old ones replaced by the new ones in search engine indices.

So to conclude, 404 errors are bad from both, the users and the search engines.

Hope it helps my friend.

Best regards,

Devanur Rafi

Hi, can you please explain the question a bit more clearly? However, let me try to address it:

To view the search volumes for keywords/phrases in Google from so and so geo location, you can use Google Keyword Planner, here:

https://adwords.google.com/KeywordPlanner

You need to login to your Google account to be able to use the keyword planner.

Once you have your master list of keywords/phrases based on relevance and search volume, you then need to think about keyword mapping which is a process that involves, coming up with keyword/phrase specific pages to be added to your website targeting each one of these keywords/phrases. So you will be mapping these keywords/phrases to your URLs (pages). Here is an example:

Suppose your list of keywords has a key phrase, "mutual funds". You can map it to a page URL like:

yourdomain.com/mutual-funds.html

Likewise, you beef up the size of your website with pages targeting keywords that resulted from your keyword research and analysis using tools like Google Keyword Planner etc.

Once you are done with keyword mapping process, you get down to write the content for each of these pages. You also need to optimize the on-page SEO elements on these pages like, the page Title, Description, H1 tag, content itself, contextual linking (internally to other pages) from within the content etc.

Please feel free to elaborate your query so that we can address it more accurately.

By the way, SERPs means, search engine results pages, the ones that Google or any other search engines give us when we search.

Best regards,

Devanur Rafi

No problem my friend : -))

He is trying to consolidate or find the total number of landing pages that do not have any traffic at all. So, screaming frog seo spider can be used to crawl the entire website (with the substring in the URLs) and substitute the URLs that have driven at least 1 visitor. He is not trying to get a hold of his historic or old analytics data. The question is pretty straight forward unless I missed something.

Hi, only one version, either the non-www or www should throw an HTTP header status 200. Suppose you prefer the www version then you should ideally have the non-www version redirected to the www version via a server-side 301, permanent redirection. You can also set the preferred domain setting to the www version in Google webmaster tools account. With 301 in place, you should see things falling on the right track. Care should be taken that only the preferred version URLs are used for any and all of your campaigns as well as link building or link earning purposes.

Best regards,

Devanur Rafi

Hi, you can use a crawler like Screaming Frog SEO Spider to come up with total number of pages with some unique string in the URLs, substitute the URLs that have traffic from these and the rest will be ones with no traffic.

You will have to use the paid version of Screaming Frog SEO Spider if you want to crawl more than 500 pages and here is the section of the user guide that tells you how to do a regex crawl:

http://www.screamingfrog.co.uk/seo-spider/user-guide/configuration/#9

Best regards,

Devanur Rafi

Hi Fred,

Please wait for at least 2 weeks for the change to reflect in Google. This happens and depends on how popular your site is in terms of link profile, DA, PA  etc..I still see "http://weekend.visitsweden.com/no" (without trailing slash) in Google's index. Let us wait for sometime. Nothing to worry about it.

Hi, you should initiate an action against them but before doing that, have a word with them and ask them to take it down. They are trying to hijack your prospect visitors by showing your content in an iframe of their website. This is a known dirty trick. By the way, when I clicked the click you mentioned, it took me to a download and probably an adaware of malware.

Please do not keep quite and act now to have that malicious site to stop showing your content. Also, they are eating up your server's bandwidth. You can even complain this to the other website's hosting provider.

Good Luck my friend.

Best regards,

Devanur Rafi

Hi Fred, now its perfect. It should soon reflect in Google and you will be able to see it in site: search. Good Luck my friend.

Best regards,

Devanur Rafi

Moz does not provide backlink indexing service.

Hi Surabhi,

If you are serious about getting your backlinks indexed fast and for sure, you need to make use of paid services like:

1. http://linklicious.co/

2. http://www.lindexed.com/

3. http://backlinksindexer.com/

4. http://www.instantlinkindexer.com/

5. http://www.indexification.com/

Free pinging tools are not so efficient compared to the above.

Best regards,

Devanur Rafi

Hi,

Hope these will explain about DA and PA along with the factors that influence them:

http://moz.com/learn/seo/domain-authority

http://moz.com/learn/seo/page-authority

Best regards,

Devanur Rafi

Dear Fred,

Here is your redirection setup:

http://weekend.visitsweden.com/      being redirected via 301 to

http://weekend.visitsweden.com/no    being redirected via 301 to

http://weekend.visitsweden.com/no/

So, I would suggest you to remove the interim URL without the trailing slash after 'no'. Let the original homepage, http://weekend.visitsweden.com/ also be redirected to http://weekend.visitsweden.com/no/ (the one with trailing slash) via 301.

So your redirection setup should be as follows:

http://weekend.visitsweden.com/  - via 301 to - http://weekend.visitsweden.com/no/

This should fix the issue. Essentially, we are eliminating the redirection loop here.

By the way, both the URLs, with and without trailing slash appear in Google when searched with the following queries:

weekend.visitsweden.com/no

weekend.visitsweden.com/no/

Best regards,

Devanur Rafi

Oh, ok. If that's the case, pls don't worry about those in the index. You can get them removed using remove URL feature in webmaster tools account.

Hi,

You should not nofollow links to your social profiles. Your social profiles can benefit from your links.

This has been answered here on Moz:

http://moz.com/community/q/is-it-better-nofollow-or-follow-links-to-external-social-pages

Best regards,

Devanur Rafi

Hi,

I 100% agree with FedeEinhorn.

roofrackworld.com.au seems penalized somewhere around November 2013 to December 2013.

Go here: http://www.barracuda-digital.co.uk/panguin-tool/

Give an offline access to the tool for your Google Analytics account. Select the date range from September 2013 till date. Look for any Google update related penalty.

Please post your observations here so that we can take it from there.

Best regards,

Devanur Rafi

Hi,

There can be many on-page issues but here you go for few of those..

1. Content of at least around 500 words per page, the more the better. Care should be taken its unique, highly relevant, up-to-date, useful to your visitors. If substantially similar or identical content has to go in two pages with different URLs, considering going for rel=canonical implementation to avoid duplicate content issue. See to its the keyword density (though there are no thumbrules) is around 1% in the article body. This is apart from other indexable content on the page like, page title, description tag, image alt text, image titles etc.

2. Page Title - the essence should be captures in the first 55 characters including spaces with your target term for the page towards the beginning

3. Description meta tag - should be around 150 characters including spaces. Description tag, though its not one of the parameters for ranking, it can have dramatic impact on your CTR from the SERPs.

4. The H1 tag - Only one H1 tag should be there per page with the target term in it

5. Links from within the content pointing to other internal pages with relevant anchor text.

6. Make sure your servers are fast enough with no down times to the possible extent. As you know page loading time is an official ranking factor.

7. Make sure to keep your URLs short to around 75 characters. Shorter, crisp, URLs that convey clear hierarchy to both the search engines and users tend to out-rank lengthy ones. URLs should include your keywords/phrases and hyphens to be used as word separators.

8. Easy and clear website navigation - care should be taken that no graphics or images are used for internal navigation, especially in the top navigation menu. The internal navigation should be only through text and JavaScript based navigation should be avoided by all possible means. With Flash and JavaScript off, users should be able to navigate through the website without any issues. To the possible extent, all the pages should be reachable within 2 to 3 clicks from the root folder. If this is not possible, add the deep nested pages to the Sitemap.xml file so that they can be found by the search engines without any effort.

9. Frames and Tables - avoid the usage of Frames and tables. Div tags can be used instead of tables as Divs result in faster rendering and provide better mobile experience also.

10. Cleaner code - the code on the website should be free of errors to the possible extent. A website with clean code will have an edge over others with coding errors. bbc.co.uk can be your benchmark in this regard. You can use http://validator.w3.org for the purpose.

11. Consistent and optimal performance on all device platforms -  care should be taken so that the pages load and render well on all platforms like, Desktop, Smartphones and Tablets. You can consider responsive web design here.

12. Social media sharing - all the important pages on the website should have socialsharing buttons for the users to be able to share the pages on Facebook, Twitter, Google Plus etc.,

13. 404 error-handling mechanism - for all the non-existing or unavailable resources in the website, a proper custom error document should be in place that enables the visitor to further navigate through the website. All 404 errors should throw a HTTP header status message 404 and not any other header status message.

14. Make sure that your website does not have any orphan pages. Every page on the website should be reachable by at least one text link.

Also, please make it a routine to perform keyword research & analysis at regular intervals to come up with a list of new or trending keywords/phrases in your niche so as to add corresponding new pages to your website.

Please keep a watchful eye on the Webmaster tools account for any error like 404, 500 series, manual actions etc.

So those were few items that you can look in to for making your website perform better in the search engines.

Hope it helps my friend.

Best regards,

Devanur Rafi

Hi,

Please try this and let us know the results:

Suppose this is one of the pages in discussion:

http://www.yourdomain.com/blocked-page.html

Go to Google, type the following along with double quotes. Replace with the actual page:

"yourdomain.com/blocked-page.html" -site:yourdomain.com

Anderss has pointed out to the right article. With robots.txt blocking, Google bot will not do the crawl (link discovery) from within the website but what if references to these blocked pages are found else where on third-party websites? This is the case you have been into. So to fully block Google from doing the link discovery and indexing these blocked pages, you should go in for the page-level meta robots tag to block these pages. Once this is in place, this issue will fade away.

This issue has been addressed many times here on Moz.

Coming to your concern about the crawl budget. There is nothing to worry about this as Google will not crawl those blocked pages while its on your website as these are already been blocked using robots.txt file.

Hope it helps my friend.

Best regards,

Devanur Rafi

Hi,

I would look at this from two perspectives.

1. These thins pages could have been beefed-up with some unique content or at least the content could have been re-written to make them unique. Personally, I prefer to make the duplicate pages unique instead of deleting them.This of course depends on the number of pages and the level of duplication.

2. Now that these pages have been removed from the website, you should be erasing all the links to these from within the website from all the places like, sitemaps and internal linking so that the search engines do not find a link pointing to them that might end-up in a 404 error. You should also consider if there have been any references left to these pages from third-party web properties.

Best regards,

Devanur Rafi