Welcome to the Q&A Forum

Waiting on an answer from our dev team on that now.  In the meantime, here's what I can tell you:

• Number submitted in XML sitemaps per GWT: 13,882,040 (number indexed: 13,204,476, or 95.1%)

• Number indexed: 17,182,818

• Difference: 3,300,778

• Number of URLs throwing 404 errors: 2,810,650

• 2,810,650 / 3,300,778 = 85%

I'm sure the ridiculous number of 404s on site (I mentioned them in a separate post here) are at least partially to blame.  How much, though?  I know that Google says that 404s don't hurt SEO, but the fact that the number of 404s is 85% of the difference between the number indexed and submitted is not exactly a coincidence.

(Apologies if these questions seem a bit dense or elementary.  I've done my share of SEO, but never on a site this massive.)

Following up on this post, I did a pretty deep dive on our log files using Web Log Explorer.  Several things have come to light, but one of the issues I've spotted is the vast difference between the number of pages crawled by the Googlebot according to our log files versus the number of pages indexed in GWT.  Consider:

• Number of pages crawled per log files: 2993
• Crawl frequency (i.e. number of times those pages were crawled): 61438
• Number of pages indexed by GWT: 17,182,818 (yes, that's right - more than 17 million pages)

We have a bunch of XML sitemaps (around 350) that are linked on the main sitemap.xml page; these pages have been crawled fairly frequently, and I think this is where a lot of links have been indexed.  Even so, would that explain why we have relatively few pages crawled according to the logs but so many more indexed by Google?

You might also want to see if any of your blog content has attracted any links.  If so, then it might be worth the hassle of rebuilding the content in your blog and doing 301s to the appropriate posts.

In addition, it might be worth checking to see if you can export the WP archives in a format that could be imported by Weebly.

Due to some questions over direct traffic and Googlebot behavior, I want to do some log file analysis.  The catch is this is a Mac shop, so all our systems are on OS X.  I have Windows 8 running in an emulator, but for the sake of simplicity I'd rather run all my software in OS X.

This post by Tim Resnik recommended Web Log Explorer, but it's for Windows only.  I did discover Sawmill, which claims to run on any platform.

Any other suggestions?  Bear in mind our site is load balanced over three servers, so please take that into consideration.

I was actually thinking about some type of wildcard rule in htaccess.  This might actually do the trick!  Thanks for the response!

So I’m working on optimizing a directory site.  Total size: 12.5 million pages in the XML sitemap.  This is orders of magnitude larger than any site I’ve ever worked on – heck, every other site I’ve ever worked on combined would be a rounding error compared to this.

Before I was hired, the company brought in an outside consultant to iron out some of the technical issues on the site.  To his credit, he was worth the money: indexation and organic Google traffic have steadily increased over the last six months.  However, some issues remain.  The company has access to a quality (i.e. paid) source of data for directory listing pages, but the last time the data was refreshed some months back, it threw 1.8 million 404s in GWT.  That has since started to grow progressively higher; now we have 2.3 million 404s in GWT.

Based on what I’ve been able to determine, links on this particular site relative to the data feed are broken generally due to one of two reasons: the page just doesn’t exist anymore (i.e. wasn’t found in the data refresh, so the page was simply deleted), or the URL had to change due to some technical issue (page still exists, just now under a different link).  With other sites I’ve worked on, 404s aren’t that big a deal: set up a 301 redirect in htaccess and problem solved.  In this instance, setting up that many 301 redirects, even if it could somehow be automated, just isn’t an option due to the potential bloat in the htaccess file.

Based on what I’ve read here and here, 404s in and of themselves don’t really hurt the site indexation or ranking.  And the more I consider it, the really big sites – the Amazons and eBays of the world – have to contend with broken links all the time due to product pages coming and going.  Bottom line, it looks like if we really want to refresh the data on the site on a regular basis – and I believe that is priority one if we want the bot to come back more frequently – we’ll just have to put up with broken links on the site on a more regular basis.

So here’s where my thought process is leading:

• Go ahead and refresh the data.  Make sure the XML sitemaps are refreshed as well – hopefully this will help the site stay current in the index.
• Keep an eye on broken links in GWT.  Implement 301s for really important pages (i.e. content-rich stuff that is really mission-critical).  Otherwise, just learn to live with a certain number of 404s being reported in GWT on more or less an ongoing basis.
• Watch the overall trend of 404s in GWT.  At least make sure they don’t increase.  Hopefully, if we can make sure that the sitemap is updated when we refresh the data, the 404s reported will decrease over time.

We do have an issue with the site creating some weird pages with content that lives within tabs on specific pages.  Once we can clamp down on those and a few other technical issues, I think keeping the data refreshed should help with our indexation and crawl rates.

Thoughts?  If you think I’m off base, please set me straight.  

In the interest of determining why our ecommerce site isn't converting, I've been spending some quality time with GA.  I've suspected that our front page is part of the problem, especially where our organic traffic is concerned (we get a good deal of referral traffic from a link on an OEM's site).

According to the Behavior Flow report under the Behavior section of GA, organic traffic to our home page is hemorrhaging (roughly 60% bounce rate).  But when I went to the All Pages report (Behavior > Site Content > All Pages) and looked at organic traffic to our home page, then looked at the Medium as a secondary dimension, I'm getting a bounce rate of 35%.  Why the massive discrepancy?  Can somebody assist?

It actually wasn't a manual edit.  For rel=publisher, I went to a Titles & Metas under the Yoast SEO plugin, went to the Home tab, and entered the company's Google+ page there.  For authorship, I went to the author's profile page in WordPress and added their Google+ page in the appropriate field.

It sounds like this is going to take some template editing.  I'll give it a try when I can.

Thanks!

I successfully enabled rel=publisher on our WordPress blog, and as a test I also enabled rel=authorship for a set of blog posts.  (Tested both in Google's Rich Snippets Tester.)  However, on the individual blog posts the publisher credit disappears.  Is there a way to enable both to appear on blog posts?

So I started at my company around six months ago.  I claimed the company's Google+ page, and have been actively updating it.  I also took the step of claiming the company's location in Google Places as well (yes, I know Google is shutting down Google Places in favor of Google+, but I figured it was a logical step to take anyway).

Here's my dilemma: the company moved locations 2-3 years ago.  That old location is still appearing in Google Places, and it also has its own Google+ page.  Is there a recommended way to redirect that old location to our new office in Google Places and Google+?

Assuming this is on-page copy and not a title tag, then I'd say you're fine.

Assuming your syntax is correct (I think it is, based on 301s I've done in the past), here are my tentative responses:

1. It's generally best to redirect to links that are as close to the original in terms of content.  If you don't have a potato mixers page, I'd suggest building one and pointing the old link to it.
2. The first line shouldn't override all the others.  You're saying the domain (i.e. the home page) should be pointed to home-appliances.  The other two redirects deal with different links.  So you should be okay.  However, htaccess is tricky, so as is always the case with 301s, test the heck out of 'em when you implement them.

Good luck!

If you're talking about enabling compression in htaccess, then I'd say absolutely.  Google has made it pretty clear that page speed is looked at as a signal.  I've used gzip in the past.  Use Google's page speed tool in Firebug.

So you inherited a dinosaur of a website and are expected to get organic traffic.  Oh, the humanity...

First off: yes, tableless CSS sites are definitely considered best practice.  They tend to be lighter overall (i.e. pages generally contain less code), meaning they load faster and Google can crawl them faster, both of which are search signals.  BITD Amazon's pages were crammed with nested tables.  Today, not so much - it's taken them some time and effort to get there, but a lot of their content is now table-free.

However, please understand that when you say "converting the ENTIRE website into Divs," you are ultimately talking redesign, with the potential hassle and expense that entails.  Is it worth it?  At the risk of sounding really indecisive: maybe, maybe not.  It depends on a range of quantifiable and subjective factors: how much content really needs to be rewritten, how long would it take, how competitive is the site at present, how old is the current design of the site, etc.

The wider issue here is on-site vs. off-site SEO.  On-site is still very important.  On-site by itself is not enough, which you seem to understand (hence, your question on link building).  However: without a good foundation through a properly optimized web presence, link building won't be as effective.

Here's what I'd recommend.  I did a static build of a site for a client - quick job, small site (i.e. less than 10 pages).  Around 6-12 months later, we moved the site into WordPress.  It was quick, painless, quite affordable to the client, and the client now has a blog and can update/add pages with no involvement from a web designer, which suits me just fine.  However, I had already optimized the site quite effectively beforehand, so it wasn't a big deal, and as I say, it was a small site.

In your case, if:

• the site is already well optimized (good title tags/meta descriptions, well written content)
• it has a good, attractive design
• it's a mature site (URLs have been indexed for a while)
• it isn't a crapload of pages that you would have to migrate

...then it might be worth the time and expense to move the site into a content management system.  I'm a huge fan of WordPress, but there are other open source systems out there that are perfectly acceptable.

Bottom line: for any major SEO campaign, I strongly believe you need the right foundation in place first in the form of a strong website.  Depending on your situation, it may or may not make sense to move it into a CMS.

Hope that helps.  

If it's a brand-new web presence, generally Google and Bing will discover and crawl it fairly quickly.  However, if you want to speed up the process, the quick and dirty way is to submit to Google and Bing here:

http://www.google.com/submityourcontent/

https://ssl.bing.com/webmaster/SubmitSitePage.aspx

You'll need to have logins with Google and Bing respectively.

However, the best way to do takes a bit more work, but is far more effective.  Validate your site in Google Webmaster Tools and Bing Webmaster Center:

http://www.google.com/webmasters/

http://www.bing.com/toolbox/webmaster/

Submit XML sitemaps with both engines.  Let things run their course, check back in a week or so, and you should be indexed.

Based on what you're saying, you've done a lot of on-page optimization, which is good.  But the thing to remember is that on-page optimization, while important, is usually not enough by itself to deliver rankings.  If you've done the on-page work, it's time to do off-site optimization, which means linkbuilding.

Here's what I would suggest:

• Identify the page on your site that you want to see rank for your keyword.  Try to point links directly to that page where possible.
• There's some low-hanging fruit internally.  Notice you have no internal links?  I think internal links have a lot of unsung value - basically, you pass link credit back and forth to yourself.  You say you have lots of content?  Start pointing links from your pages to one another.
• Do you have a blog on your site?  If not, slap one up and start blogging.
• You know the keyword you're chasing.  Select blogs in that vertical and offer to do some guest blogging.

Remember: On-page optimization is the floor, not the ceiling.  It gets your site to a minimum level of acceptability, but on-page alone isn't enough.

I think what you're really talking about is crafting an SEO strategy.  I'm a big fan of persona-driven strategy development.  Go here:

http://www.seomoz.org/blog/the-8step-seo-strategy-step-1-define-your-target-audience-and-their-needs

This is the first of eight posts from a former Yahoo Media on SEO strategy.  It's going to force you to carefully consider your target audience, do a lot of competitive research, think about content, etc.  Yes, it's a lot of reading, and a lot of effort.  It's worth it.  Trust me.

The quick-and-dirty solution may be to implement a 301 wildcard redirect for the domain and any pages on that domain.  However, that is definitely the ugly way to do it.  Can you tell us what editing platform you're using?  Is it an open source platform?  If so you may be able to find a plugin that will allow for manual 301 redirects.