It's hard to say without knowing exactly what's been changed on the site/inbound links, but I wouldn't be surprised if it had something to do with the backlinks... this graph does not look normal and it appears a lot of backlinks have disappeared in July - https://ahrefs.com/site-explorer/overview/subdomains/?target=www.weddingphotojournalist.co.uk

Edit - make sure to select the "One Year" tab for the graph

Here are some options for closing the loop between web leads and closed sales -

1. Capture visitor data using the GA cookie and pass it along with each form submission. This is against GA's TOS so do it at your own risk but here is a blog post telling you how to do it - http://cutroni.com/blog/2009/03/18/updated-integrating-google-analytics-with-a-crm/. (The author of this blog post actually works for Google Analytics so it's kinda funny he tells you how to break their TOS...)

2. Use a contact form that automatically tracks conversions for leads, such as Convertable - (full disclosure - I'm a co-founder of Convertable but it solves the issue you are having). You can also try other similar paid services like HubSpot, Salesforce, BestContactForm.com, etc.

I highly recommend checking out Long Tail Pro. It uses the GAKP but its interface is much easier to use rather than using Google directly. Also, you can plug in a couple of keywords and generate thousands of long tail keyword phrases and filter by search volume, organic competition, keyword difficulty (uses the Moz API), etc.  While the data is still from Google and the keyword data may not be 100% accurate, I think it will get you a lot closer to what you are trying to accomplish than you would using any of these other tools.

I'm a big fan of testing keywords in AdWords before choosing the ones you want to use for SEO. You can set up the AdWords campaign today and get actual data on the number of impressions (number of times the keyword is searched), CTR and conversions generated by each keyword in only a few hours. I'm not sure how familiar you are with AdWords but it is absolutely critical that the campaigns be set up correctly from the get-go, otherwise you will get inaccurate data and may drain your budget quite quickly.

Since your goal is to test the keywords for SEO, make sure you are only using [Exact Match] keywords. And it is probably wise to uncheck the option to "Include plurals, misspellings, and other close variants."

Also, make sure you are not opted into the Display Network but rather just the Search Network (ideally just Google, not the search partners as well).

Finally, I'd suggest using a tool called LongTailPro to instantly come up with thousands of long tail keywords that you might not have considered yet. You can sort/filter by CPC, advertiser competition, organic competition, keyword competitiveness, search volume, etc. It gets the keyword data from Google and also uses the Moz API to check organic competition. It's definitely my favorite keyword research tool and one that I don't ever really see talked about on Moz.

Also, have you tried "Fetch and Render" as Googlebot in GWT? Is the rendered page in GWT the correct Converse page or is it still the wrong URL?

The Google cache date is June 17th so something is obviously causing Googlebot to associate the Converse page with that particular URL. When did you do the "pop in" for the Converse page? So when users went to http://shop.nordstrom.com/c/converse, they were displayed some kind of pop up banner ad?

Any chance you can share the URL here or via PM? Also, for the fake page showing up in GWT as the top priority, did you click on the link to see more details? Such as where it is being linked from? That might provide some insights as to why Googlebot is still trying to access that page.

I just wanted to chime in that Convertable provides multi-touch attribution modeling. When a user submits a lead through the contact form, you will be able to see how many visits the user made to the site, date and time, the source of every visit, every page they viewed on each visit, how much time they spent on the site for each visit, and so on. This information allows you to determine which traffic sources, ads, keywords, etc. should actually be credited with the sale at the end of the sales pipeline.

Yeah, it's definitely still in place.

But I don't really understand why, especially since Google loves to track everything everyone is doing but doesn't want to allow anyone else do it. Also, I'm not sure how much they enforce it, especially because Google Analytics' own Justin Cutroni has written about how to capture personally identifiable information in GA many times in the past -http://cutroni.com/blog/2009/03/18/updated-integrating-google-analytics-with-a-crm/. (Although he includes a disclaimer about the privacy policy, he still writes the post...)

Anyways, Keri is right that the policy is still in place so if you rely heavily on Google Analytics for your site, it's probably best not to blatantly violate their TOS since we all know Google can and will bring the hammer down anytime they feel like it.

I'm actually a co-founder of Convertable. We created Convertable to do exactly what Ricky is looking to do because we ran into the same problem not being able to use Google Analytics or AdWords conversion tracking to identify the source of any individual lead. And to add onto Guillermo's comment, yes, you can definitely track which specific AdWords campaign, ads, keywords, landing pages, etc. generated each lead by using our Campaign Tracker tool.

Please let me know if anyone has any more questions or suggestions for things to add to Convertable since we are always looking to improve the service for our users!

• Patrick

Have you tried using ahrefs.com? It seems to update its index on a daily or almost daily basis, which is much faster than OSE and GWT.

Hmmm, I still don't know why your account would have gotten flagged simply for "being about SEO" so something is definitely up... I would suggest submitting your story to the big SEO/PPC news sites like searchenginewatch.com, searchengineland.com, searchenginejournal.com, etc. because people need to be aware of this and then maybe you will get a more detailed response from Google.

Where are the 301s taking Googlebot on those IP addresses? And are they the same IP addresses every time? Have you narrowed those IP addresses down to any particular datacenter/country? It could be possible there is some configuration with your server that treats IP addresses differently depending on the country... it could also be that the IP addresses getting the 301s are known blacklisted spam IP addresses but are masking themselves as Googlebot so your server's blacklist software is keeping them out. It's really hard to say without looking into the data myself but I'm definitely interested in what you find out.

What did the ads say? And the landing pages? There are tons of AdWords advertisers that bid on SEO-related keywords, so this is the first time I've ever heard of anything like this.

Actually, you are correct. Google's PageRank algorithm has always been all about links. Check it -

PageRank works by counting the number and quality of links to a page to determine a rough estimate of how important the website is. The underlying assumption is that more important websites are likely to receive more links from other websites.

More on http://en.wikipedia.org/wiki/PageRank.

This being said, Google's toolbar PR is not nearly as influential as a ranking factor as it has been in years past.

If there is no relevant new location to 301 to and you are dealing with way too many files to individually 301 them, then you can simply delete them so Google will encounter the 404 error page and remove those files from the index. That being said, if there are any old files that actually rank for search queries and generate organic traffic to your site, then I would recommend at least 301'ing those if possible so you don't lose the traffic as a result of the page being removed altogether.

If you want to track an individual lead all the way through the end of the sales funnel and tie it back to the original traffic source to determine ROI, feel free to check out our tool Convertable. It's currently in free beta but we will be launching premium features soon. We created it because there was really no easy way to look up a specific individual lead within Google Analytics and tie it to the end of the sales pipeline. Please let me know if you have any questions or suggestions for how to make it better.

Otherwise, you can try manually setting up a similar configuration by extracting data from the Google Analytics cookie and passing it along to your CRM - http://cutroni.com/blog/2009/03/18/updated-integrating-google-analytics-with-a-crm/

I'd suggest doing a 301 redirect rather than simply deleting the pages. The 301 redirect will tell Google to stop indexing the old version and index the new version instead.

Yup, that's exactly what I do for my Twitter account that automatically posts questions from the Moz Q&A (https://twitter.com/seoqa).  I have to use a third party tool to create an RSS feed since Moz doesn't have one (that I'm aware of) and the tool's free plan can only post every few hours which can get pretty annoying when it blasts 8 tweets in a row... but if you already have an RSS feed, you can use IFTTT to check your feed for new items more frequently and post to Twitter immediately.

Edit - just realized my Twitter account stopped tweeting out Moz questions on April 27th, likely as a result of Moz changing the structure of their Q&A posts and/or something broke with my RSS tool but nothing should be wrong with IFTTT and/or auto-posting to Twitter...investigating now!

Yup, IFTTT is the way to go. I use it for several Twitter accounts.

(Also, I had written a more detailed reply but I got an error message from "incapsula" and then I lost my response, ugh. So maybe Moz should look into this...)

moz.com -

Access Denied

Error code 15

This request was blocked by the security rules

I think there's a problem with the Redirect and RewriteRule functions clashing. Try changing your redirects to RewriteRule. Let me know if this works.

RewriteRule ^armaflex.html$ http://www.just-insulation.com/002-brands/armaflex.html [R=301,NC]

Are these 100+ visits from 100+ different users? It could be just one user who initially came to the site for that keyword and kept coming directly back to the site but GA is counting it as another visit for that keyword.

I would just use a sitemap.xml file instead for Google, Bing and Yahoo. Then you can submit the sitemap.xml file within the Google Webmaster Tools and Bing Webmaster Tools (includes Yahoo). You can easily create an XML sitemap at http://www.xml-sitemaps.com/

Great call on both the canonical tags and breaking out of the frame. I imagine the site will be indexed properly within a week using these tactics.

Interesting, I am now seeing what you are talking about...

Can you share your URL so we can take a look? Feel free to PM me if you want to keep it private.

Hi there,

We had a client with the same issue. They could use Google Analytics to track which traffic sources generated leads, but there was no way to track which lead came from which traffic source.

So we created a tool called Convertable (http://convertable.com) that does just this. It emails you the lead information with full analytics data attached to it. Then you can update the status of the lead in the online dashboard so you can enter in how much revenue, how many shirts, etc. were generated by each individual lead. This will enable you to tie the amount of revenue generated back to the original traffic source in order to determine the true ROI of online marketing campaigns. Convertable is still in free beta mode but we will soon be launching premium plans with more features. Feel free to check it out and let me know if you have any questions or suggestions for how to improve it!

• Patrick (co-founder of Convertable)

Can you share the URLs here? Or feel free to PM me the URLs and I can take a look.

Hmm, I am seeing your original page rank #7 for "iphone 5s abonnement" here in the US on Google.com and then #3 when I try searching on Google.nl... are you still seeing them at #33?

And, do the include pages have any original content or are they exact duplicates? Rather than redirecting them, I might suggest putting original content on them in order to rank for each individual product.

It's hard to say without seeing your site and seeing how the virtual includes are set up, but if you are still having issues ranking with only the pages that use the virtual includes, then it might be a good idea to set them up as 301 redirects. But again, I can't say for sure unless I saw the site, can you share it here?

Also, it is worth mentioning that it depends on what Google Analytics report you are looking at. Sometimes the value will be set to "(not set)" if you are looking at the keywords that drove traffic to the site for each traffic source, and in that case, any visit that did not come from a keyword will be (not set), such as referring sites since they obviously did not come from a keyword...

Hmm, typically (not set) means AdWords is not correctly linked to the Google Analytics account. But as you said, you are not doing AdWords or any paid marketing. Can you dig in to see what the landing page was? Or where the visitor was located? That may help clear up some questions. Also, here is more information from Google - https://support.google.com/analytics/answer/2820717?hl=en

The rankings can vary person to person depending on several different factors. So just because you are not seeing your site in the rankings, it is still possible that other people are seeing it and that's why you are getting visits for those keywords. I would check your Google Webmaster Tools Search Query report for more information.

Are you still having issues with this? The .htaccess file is quite powerful and can easily bring down a site with even the smallest mistake. Let me know if you are still having problems and I will try my best to help you out today.

Hi there!

I just searched Google for site:http://traditional-cleaning.co.uk and I see 26 pages indexed, so it is good news that your site is being indexed! Google considers the www version and the non-www version of the URLs to be different sites, so it's best to pick one and stick with it. You can do this by 301 redirecting all of your URLs that start with "www" to the root. I usually recommend using an .htaccess file for this as long as your server is compatible. Here is some more information - http://301redirects.net/non-www-to-www-301-redirect-urls.php

You will also want to redirect the index.htm file to the root URL. Here is how you can do this (with .htaccess) -

RewriteEngine On
RewriteBase /

redirect html pages to the root domain

RewriteRule ^index.html$ / [NC,R,L]

These two steps should resolve your issues with both Google and Moz. I also highly recommend using Google Webmaster Tools and designating your preferred URL just to be safe. Let me know if you need any other help!

Hi there,

I am doing a crawl on the site listed in your profile (www.abdserotec.com) using Screaming Frog SEO Spider using Googlebot as the User Agent, and I am seeing many more URLs than the 14,000 pages you have. The bulk majority of these excess pages are the Search Results pages (such as http://www.abdserotec.com/search.html?searchType=BASIC&searchTerm=STEM CELL FACTOR&cat=&Filter2Name=GO&Filter2Value=germ-cell development&filterCount=2&type=&filter1name=Spec&filter1value=STEM CELL FACTOR). While these URLs are not showing up in the Google Index when you try searching your site with the site: command, Google is still definitely accessing them and crawling them. As Tuzzell just suggested, I also highly recommend configuring the parameters within GWT.

I've been having the same issues and it's getting to be pretty frustrating!

I'm a bit confused. Didn't blocking the test subdomain with the robots.txt already accomplish what you are trying to do? Or are the test pages still somehow indexed? Or is your main site affected by the robots.txt? Anyway, I would suggest using the .htaccess file to block search engines from accessing the subdomain rather than a firewall - http://stackoverflow.com/questions/6738896/excluding-testing-subdomain-from-being-crawled-by-search-engines-w-svn-reposit

When you used the old GAKT, did you specify Exact Match? If not, then that's what's happening here. The Keyword Planner only displays Exact Match by default, so the numbers are going to be much, much lower than Broad Match using the old GAKT.

As long as you have the canonical tags set up correctly, then you don't need to do anything else with the URLs that have parameters in them. Google will eventually start dropping those URLs from their index.

The rel="canonical" tag needs to have the URL that you wish to be the primary designated URL. This tells the search engines only to index this version of the URL and no other versions, and it also tells the search engines that links to other versions of the URL need to be counted as pointing to the canonical version.

So if your page is www.mysite.com/boats.php and you use www.mysite.com/boats.php?utm_medium=adwords, etc. then the canonical tag should be set to www.mysite.com/boats.php so the search engines don't bother messing with the URLs with the parameters in them.

Absolutely keyword data!

Also, I've always wanted a better third party traffic measuring system. Everyone here knows that the services like Alexa, Compete, Quantcast, etc. all vary widely in the numbers they report and are oftentimes woefully inaccurate when comparing the data to a site's internal analytics. And considering these companies are able to make millions off of their services, I think it would make complete financial sense for Google to enter this space. And I have no doubt they could dominate the market immediately since they pretty much know everything about every person as well as the majority of websites, so their data would be much more accurate and reliable than anyone else's.

Definitely Long Tail Pro. It uses the GAKT to pull in keyword data but it is much more powerful than just using the GAKT for organic keyword research. You can enter in 4 or 5 base keywords and it will automatically generate thousands of related keywords, which you can then filter and sort by a number of metrics, including organic competition which uses the SEOmoz Keyword Difficulty API. It's by far my current favorite organic keyword research tool.

Since the "important pages" have been removed "by request" leads me to believe somebody with access to your GWT has manually requested to remove the site from Google via the URL removal tool.  You should be able to see who has access to your GWT and who removed the URLs within GWT. Just go to "Google Index" then "URL Removal" then click on the select box to the right to see the URLs that have been removed by others. You should be able to resubmit the site/URLs to Google afterwards.

When were the products on GMA and when did you see the rankings drop? There have been several algorithm updates recently so it's possible that the drop in rankings is unrelated to the GMA appearance and it's simply a coincidence. A mention/link from GMA should definitely be helpful for SEO, and I have not heard of "temporary freezes" put on sites before due to a big increase in clicks. Also, the backlinks and internal links might not have shown up yet in GWT but they should eventually (also depending on when the products were on GMA).

For a brand new site, I would start out with targeting the less competitive keywords as your top priority so you will hopefully see some progress rather quickly. If you start out focusing on the highly competitive keywords, you might get discouraged since it is awfully hard to rank for competitive keywords with a brand new site. I would suggest gradually moving onto the harder and harder keywords as your site grows and establishes itself as an authority down the road. If it is not feasible to target some of your phrases with individual product pages, then I highly recommend adding a blog to the site to target these keywords. Good luck!

There shouldn't be any problems with the robots.txt file as far as blocking the homepage in Google, especially since the homepage is indexed in Bing and both crawlers abide by the robots.txt.

Did you change domains when you did a site redesign? I noticed your sitemap has URLs from a completely different domain name so that may have something to do with it - http://www.truebluelifeinsurance.com/sitemap.xml. I'd make sure to change the sitemap and then set your preferred domain name in Google Webmaster Tools.