Welcome to the Q&A Forum

Is your title tag well written? Is it around 60 characters? I would make sure the title tag is a good description of the page/product, otherwise Google might try to create better, longer title tags for your pages.

Can you share the URL and the search result? That will help in solving your problem

Mark

searchmetrics.com can also provide you with a bunch of this data - you can usually see where a competitor ranked for a keyword historically over time - I would check them out - I find it very helpful for competitor analysis

Ok - so a few things - I'm not worried about the sitemaps - those aren't a big deal (should be fixed, but not first priority).

In my eyes, you need to do 2 things pronto:

1 - remove the old.toughtimeslawyer subdomain - do this by blocking the subdomain via the subdomain's robots.txt - do a disallow: / in the robots file, and then go into the webmaster tools for the subdomain (which you should set up if you haven't already), and use the URL removal tool to remove the subdomain from Google. Once this is done, and the subdomain is no longer indexed, add a noindex meta tag to all pages and remove the robots.txt block - this should get rid of the subdomain permanently from the index.

2 - you need to get a handle on the 301 redirects - crawl your current site structure, and redirect any old URLs to the new URLs - you should keep a close eye on the 404 errors and soft 404 errors sections in Google Webmaster Tools, and make sure to fix those errors as they pop up. This can be time consuming, but it's key. Set up a redirection plugin for your wordpress - the redirection plugin can also track 404 errors and then you can tell them where they should redirect. While this should have been done in advance and is key to any properly planned and organized site migration, you can do this after the fact and try to catch as many of these 404s as well. I'd also look at your inbound links from OSE or some other tool, and make sure those pages are redirected to their proper location, so you preserve the inbound link juice.

These are the key things in my mind you should do according to your post. Good luck!

You can add noindex to the whole subdomain, and then wait for the crawlers to remove it.

Or you can register the subdomain with webmaster tools, block the subdomain via the robots.txt with a general Disallow: / for the entire subdomain, and then use the URL removal tool in Webmaster Tools to remove the subdomain via robots.txt. Just a robots.txt block won't work - it won't remove the pages, it'll just prevent them from being crawled again.

In your case, I would probably go the route of the robots.txt / url removal tool. This will work to remove the pages from Google. Once this has happened, I would use the noindex tag on the whole subdomain and remove the robots.txt block - this way, all search engines should not index the page / will remove it from their index.

Mark

You can try a caching plugin like W3 total Cache - http://wordpress.org/plugins/w3-total-cache/

It minifies CSS, and does a lot more to speed up your site - just read all of the instructions first - it can be pretty powerful

Good luck,

Mark

You can use scrapebox to scrape whois info for the domains, and then use the emails you get from the scrape.

For manual guess work, you can try using rapportive and plugging in variations and seeing what you find - you can also try this nifty tool built by the team at SEO Gadget to pull contact info and other various metrics - http://tools.seogadget.co.uk/

Good luck,

Mark

He has an awesome post here on Moz about matching keywords using an array formula - the info is here - http://moz.com/blog/keyword-research-using-categories

formula template is =IF(SUM(NOT(ISERROR(FIND('Keyword Types'!$A$2:$A$7,$A2)))*1)>0,P$1,"Non-"&LOWER(P$1))

then hold shift, CTRL, and enter - refer to the post for what to do - I love this method and find it very helpful for lots of stuff.

Good luck,

Mark

nope - not browser specific - cookie enabled, and in chrome

Did you make any structural changes to your site before this happened? Did the URL structure change or the main navigation structure? I would first check your robots.txt and meta robots tags to make sure nothing was accidentally changed to block the search engines/remove the pages from the index. An accidental change of a setting can often make major differences for a site.

If you can provide the URL, we can take a quick look.

Mark

Guys, I've brought this up with Moz team on Twitter - they said they're having it internally as well and are working on fixing it - hopefully will be resolved soon

That indicates the language of the page - hreflang does better, in telling the search engines these pages are related, are not duplicate content, but are the same product and targeting searchers in different languages. This is how I recommend to clients today to configure their multilingual sites - this works for folder, subdomain, or ccTLD setups

With different languages, it's fine to target them to different subfolders on the same domain - this way, you get the strength of the domain helping out each of the subfolders, and link building for one can help the other.

I would recommend in this case implementing the hreflang tag, indicating to the search engines that these pages are translations of one another and are meant to target different languages.

With your site targeting languages and not countries, I wouldn't geotarget the subfolders via Webmaster Tools, and I wouldn't necessarily go down the country specific TLD root, .com, .co.uk, .fr, etc.

I think using the hreflang tag here is your best bet - you can learn more about it here - http://support.google.com/webmasters/bin/answer.py?hl=en&answer=189077

This should eliminate any duplicate content issues and help your site's performance in both languages.

Good luck,

Mark

Depends on what you're trying to do - if you're looking at a particular page, and you want to highlight external links or followed links, use the mozbar's highlighting links function.

If you want to crawl your site and find all of the external links, I would recommend using Screaming Frog SEO Spider - http://www.screamingfrog.co.uk/seo-spider/

The extension you use to create the search engine friendly URLs should be good enough to redirect the category and product pages.

You may want to check out an extension like this - http://www.inveostore.com/magic-seo-urls-for-opencart-9 (note - I haven't used this in the past, but may solve what you're looking for - either way, talk with your site developer and make sure the extension is relevant for your version of opencart)

I can see that your site is running on Open Cart -

long term, you shouldn't lose rankings if you switch to a more friendly, navigable URL structure. You just need to make sure you have a a redirection plan put in place to redirect visitors and search engines from the old URLs to the new URLs. These redirects should be 301 redirects, so the old URLs are replaced with the new ones in the search engine indices. If you are using an extension to create the new URLs, make sure it will 301 redirect the old URLs to the new ones.

You may see a temporary drop in rankings, but if implemented properly, things should recover rather quickly.

Good luck,

Mark

If you're all good I'd mark the question as answered, and mark any answers you found helpful as good answers - this helps all of us here in the community!

Thanks,

Mark

google can read your site - they may be reading the content and changing your titles that they are displaying - instead of showing your title tag, they are showing one that they believe to be more relevant.

In Webmaster Tools, you can fetch the page as Googlebot and see what they can see - if you want me to help you via your webmaster tools account, contact me in private message and we can take it from there. But you should be able to do this on your own and see what Google can see - once you know what they can see, you can diagnose if they can actually crawl your page, or if they are rewriting the tags for other reasons.

But your 301 redirect you put in place looks ok to me.

Not sure why you're saying Google can't access your site and your pages have been removed - you can see from this search query they are still in the index - https://www.google.com/search?hl=en&source=hp&q=site%3Awww.thegrooves.net

I would definitely improve on your site - you have lots of empty pages created that are simply gallery pages, each page showing one image - I would create a gallery showing multiple images on a page - better for the search engines, better for users.

It looks like your site could use a refresh - it also doesn't look like you have Google Analytics installed  - this would be a very powerful resource for knowing who is visiting your site, what they're doing, how visitors are converting, etc. I would definitely consider modernizing and fixing up your site

Mark

Hey,

Glad to hear it worked out

Mark

Hi Ron,

I'm happy to take a look at the site here on the forum - you just need to provide us with your URL and see if we can find an answer for you why the site isn't showing up in the SERPs

Mark

Use Google Analytics for this - set up your conversions as goals, and then you can track via various reports the conversion process - what did they do, where did they go, how did they flow through your website from initial landing page to full conversion.

Google Analtyics is very powerful - it does multi touch attribution - multiple visits until your conversions, how they came each time to your site, etc.

It really is a wonderful tool - here is a guide to setting up goals - https://support.google.com/analytics/answer/1032415?hl=en

Once the goals are set up and the data is flowing, the fun really begins, and you can make lots of actionable analysis to improve your site based on it.

Good luck,

Mark

Blocking with robots.txt doesn't remove pages/files from the search engines, it only prevents them from crawling the subdomain. If they already crawled those resources, as they have in your case, the robots.txt will just block them from visiting it again, but will not remove them.

What you should do is authenticate the subdomain with webmaster tools, and then remove the subdomain via the url removal tool, as you asked. This is the way to go about it.

To verify the subdomain, there are multiple options, including via your host, or you can modify the dns to prove you own the subdomain - it really depends on your setup and what you need to do.

But to remove these results permanently, with your robots.txt block, you should do it via the URL removal tool.

Good luck,

Mark

According to what I saw here, http://www.guardian.co.uk/help/insideguardian/2013/may/24/theguardian-global-domain, they'll be working with the team at Yoast, who are some of the best in the biz at onsite SEO, so I think the Guardian should be in good hands.

OK - if this is the case, to force Google's hand (so to speak), I would canonical these brand pages over to the target pages you want your visitors to land on - this won't remove them from display on the site like a 301 redirect would, but it will point to the search engines that these pages should be shown over the brand pages in the SERPs. That's what I would do to solve your problem.

Mark

Hi Jonathan,

Could there be something technical hurting that page? A meta robots block? A robots.txt block? Perhaps overoptimization of the page?

If you can share the URL, that will be helpful to try and provide some ideas for how to fix the situation.

Mark

Based on the graph you provided, it looks like a panda hit.

I would check your landing pages and keywords reports in Analytics, and see if you can identify head terms that were hit, or if this is more a sitewide, across the board issue.

I don't believe Yoast's Wordpress SEO plugin will cover Magento files. The Wordpress SEO plugin is built into the Wordpress, and pulls the page, post, category, taxonomy, etc. data from Wordpress in order to build the sitemap. The plugin itself creates a sitemap for each page, post type, etc., and is highly configurable. It then uses a sitemap index file to connect everything together.

But since the Wordpress doesn't have the Magento information, since they are separate CMS's with separate databases, the Yoast Wordpress SEO plugin won't build a sitemap file for information in the other CMS, as it has no access to it and no knowledge of it.

You can use other tools to create sitemaps for the site, either through manual crawls or installed software on your server,  but the yoast wordpress plugin won't do it.

Mark

Thanks Paul - I can't take most of the credit - got to give it up to the guys at Screaming Frog - love the spider

You didn't provide the address of your site, but I've seen on larger sites with user generated content, or sites that have some type of forum, this type of anchor text may be indicative of a larger problem - that you have problematic pages on the site with content relating to child porn, or your site has been hacked and code/links have been injected. Thus, the spammers are linking to you with that anchor text, and your site is part of a larger chain.

I would take a close look at the landing pages for those links - are they forum pages/user profiles of spam, or are they linking to a page on your site that was hacked into.

To check if your site was hacked into, I would view the cache of Google for the page/site. If so, you can often see the stuff they have put on your page - if it doesn't show up in the regular view of the cache, try clicking the link for the text only version. While this may be negative SEO, I've seen lots more cases where this is a result of a site being hacked or user generated content sections not being moderated and which have run completely wild.

Mark

Basically, SEOMoz is crawling your site and telling you they found these errors. There errors were found by following internal links.

I took a quick look at the code on your site, and in the social icon buttons in the header of your pages at the top right, for the Twitter icon, you are linking to yourself, to piensa_piensa - here is the code

class="social-icon"><a <span="" class="webkit-html-attribute-name">target</a><a <span="" class="webkit-html-attribute-name">="_blank" href="</a>Piensa_Piensa ">src="http://piensapiensa.com/wp-content/themes/modernize-v3-11/images/icon/dark/social/twitter.png" alt="twitter"/>

Your href is piensa_piensa - this is adding this text to the URL of every page - correct this link and this error will be fixed - either use # until you link to your Twitter account, or link to your proper address of the twitter account. This will solve your problem.

Good luck,

Mark

For webmaster tools, you authenticate each site separately. These are different country level top level domains, and will be separate entries in webmaster tools. In addition, if you have different subdomains on the sites, www, my.peinsa, news.piensa, or anything else, you set those up each as a separate entry in webmaster tools.

You can geotarget the different sites/subdomains in webmaster tools as well - it really depends on the setup of your sites and the targeting you want to implement.

Mark

Hey SaraSEO,

This might be a good solution for you - I haven't tried it, but it seems like what you need - sitemap creator that can strip out parameters and follow canonical tags - http://www.inspyder.com/products/SitemapCreator/Default.aspx

Good luck,

Mark

I would try creating an open Google doc, and then listing all of the sites in the network. Kind of like the reconsideration request method, where you link to an open Google doc with all of the details of the webmasters you contacted, responses, success rate, etc

Hi Matthew,

Google has a specific form for reporting webspam - you can find the spam report here - https://www.google.com/webmasters/tools/spamreportform?hl=en

Before you submit a competitor, make sure your own site/s are clean - don't want them looking to closely into your SERPs and your sector and finding a problem with you as well.

Mark

I don't think they're "gaming" Googlebot - I think they're trying to help the bots properly crawl through the site, index the relevant content, but not create hundreds of thousands of empty pages that will simply dilute their index and lower the overall value of the site in the search engine's eyes - I think they're trying to keep the Panda hungry and not provide it with lots of yummy food for it's low quality content hungry stomach.

This is why they are noindexing the pages - not to game the system, but to actually play by the system's rules.

It looks you have a bit of redirect loop here - that might be what's hurting the server a bit

It seems like they noindexed that page because it may be part of an antiquated version of the site navigation/structure, or part of the cms and not something they want to promote. Not sure how you got there, but when you get to the primary version of a category, and then click through to the next page, the items shown change via ajax and the URL stays the same, just with a parameter that this is the second set of items being shown.

With the url staying the same, for their primary path of navigation, I don't think rel prev/next would be relevant. And these other pages probably created by the cms but not easily accessible they've noindexed - that's my best guess

You should remove the robots.txt block on the redirect - if the redirect is implemented properly, and it directs straight to the new page and not through some sort of infinite loop, it should be fine for you server to handle.

The bots think those pages are real pages, thus they have them indexed - you have blocked them from visiting the redirect and seeing that it points to another page and that the redirect page should be replaced with the real page. If things are implemented properly, they would follow the 301 redirect and remove the URL of the redirect from the search engines.

If you provide a link to one of these redirecting URLs, I'm happy to take a look and see if I can help you.

In terms of what you've written, blocking a page via robots.txt doesn't remove it from the index. It simply prevents the crawlers from reaching the page. So if you block a page via robots.txt, the page remains in the index, Google just can't go back to the page and see if anything has changed. So if you were to block the page via robots.txt, and add a noindex tag to the page, Google won't be able to see the page with the noindex tag to remove it from the index because it's blocked via robots.txt.

If you moved all of your old content to a different folder, and block that folder via robots.txt, Google won't remove those pages from the index. In order to remove them from the index, you would have to go in to Webmaster Tools and use the URL removal tool to remove that new folder from the index - if they see it's blocked via robots.txt, then and only then they'll remove the content from the index - it has to be blocked via robots.txt first in order to remove the whole folder with the URL removal tool.

I'm not sure though if this would work for the future - if you removed a folder from the index, and then added more content that was indexed previously afterwards, I'm not sure what would happen to that new content moved to that folder. Either way, Google will have to come back and recrawl the page to see that it has moved to the new folder, and then remove it from the index. So either way, the content will only be removed once Google recrawls the old content.

So I still think a better way to remove the content from the index is to add the noindex tag to the old pages. To facilitate the search engines reaching these old pages, I'd make sure there is a way the engines can get to them - make sure there is a path they can take to reach them.

Another good idea I saw on a forum post here a while ago would be to create a sitemap containing all of these old pages you have indexed and want removed. Add the noindex tag to the sitemap - using the Webmaster tools sitemap interface, you'll then be able to monitor the progress of deindexation over time - by checking how many pages on the sitemap/s of the old content are originally indexed as reported by webmaster tools, and then you can see later on how many of those pages are still indexed, this will be a good indicator for you of the progress of the deindexation.

Just an aside - you're going to have indexation issues - you have both www and non-www versions live on the site, with no canonicals pointing to one version. You also have index.php as a live page linked to from the logo. I'd definitely recommend implementing canonical tags across the site.

Mark

there are a bunch of articles out there, but each case is different - here are a few:

http://www.seomoz.org/q/panda-recovery-what-is-the-best-way-to-shrink-your-index-and-make-google-aware

http://www.searchenginejournal.com/the-holy-grail-of-panda-recovery-a-1-year-case-study/45683/

You can contact me via private message here on the forum and I can try to take a more in depth look at your site if you can give me some more detailed info.

Are you sure you got hit by Panda before we talk about a Panda hit?

Exactly - I'd build  a strategy more around promoting pages that will have long lasting value.

If you use the tag noindex, follow, it will continue to spread link juice throughout the site, it's just the individual page with the tag will not be included in the search results and will not be part of the index. In order for the tag to work, they first have to crawl the page and see the tag - so it doesn't happen instantaneously - if they crawl these deeper pages once every few weeks, once a month, or even longer, it may take a while for these pages to be removed from the index.

Hi Assaf,

(I'm not stalking you, I just think you've raised another interesting question)

In terms of index status/size, you don't want to create a massive index of empty/low value pages - this is food for Google's Panda algorithm, and will not be good for your site in the long run. It'll get a Panda smack if it hasn't already.

To remove these pages from the index, instead of doing hundreds of thousands of 301 redirects, which your server won't like either, I'd recommend adding the noindex meta tag to the pages.

I'd put a rule in your cms that after a certain point in time, you noindex those pages. Make sure you also have evergreen pages on your site that can serve as landing pages for the search engines and which won't need to be removed after a short period of time. These are the pages you'll want to focus your outreach and link building efforts on.

Mark

Link Research Tools breaks down backlinks as well and will classify links for you as toxic - see here - http://www.linkresearchtools.com/news/link-detox-clean-backlink-profile/

I haven't used this particular feature they offer, but in general, their backlink analysis and breakdown tools are very good.

Hi Jeremy,

Good question - in the crawl diagnostics, the number of links is the number of links from your page to other pages - these links can be both internal and external. It's a measure of the total number of links on your page, and tries to show how the value of your page gets diluted through your various links.

The next two columns, of page authority and linking root domains, tries to show the strength of the page in question. Linking Root Domains is the number of root domains pointing in to that page. I believe it includes both internal and external - thus, a page with 1, usually means it's only being linked to internally. On this particular crawl test, number of links is the number of OUTGOING links on the page crawled, whether those are linking to another page on your site or to external domains. Page authority is a measure of the strength of your page, on a scale of 1-100, and linking root domains is the number of domains pointing to your site.

They show you the strength of the page because if the page is strong, though there are more links on the page, this may still be ok, because the page is stronger and has more juice to spread across the site/web.

Mark

It really depends on the site - often a website will have the most links pointing to the homepage, so this is usually the strongest page. Thus, it may be easiest to rank your homepage for your primary keyword. On the other hand, if the most important keyword is a commercial term that will lead to higher conversions/sales from an inner page, like a product page, then don't forsake those higher conversions for the link benefits of the home page.

In addition, if you're going to be aggressively link building for your target term, if your inner page gets hit with a penalty, you can always 404 the page, remove the links, and start over. This is much harder to do when the overoptimized, penalized page is the homepage. So from that perspective, it may pay to optimize an inner page for your target keywords and not your homepage.

Your homepage often serves as the front page for your business - as such, it has to cater to multiple audiences, and often multiple parties who all are fighting for space on the homepage. When I worked at a not for profit museum, there were lots of different involved parties vying for space on the homepage. Thus, it may make more sense to target your primary keywords to inner pages where you can completely control your content and onsite setup.

Thus, while often it makes sense to target your most important terms to the homepage, it really depends on the site and the individual circumstances, and I wouldn't recommend targeting your homepage with your most important keywords as a blanket rule for all cases.

Good luck,

Mark