Welcome to the Q&A Forum

Awesome responses everyone - really appreciate this. It seems I have my work cut out across the various tools to try to identify root cause. At first, quick glance I don't see anything obvious popping up (for example to organic vs. direct) but I have more to check, based on the excellent input from the three of you. Thank you again!

Mark

I am seeing a pretty precipitous drop in search volume traffic (see link). My keyword rankings don't seem to have suffered too much over this period. In fact, my #1 keyword have actually increased slightly in this timeframe.

Two questions...

• Is there some way to assess overall search volume across my tracked keywords (to see if this is just a case of overall searches dropping)?
• Is there a recommended plan of attack for investigating drops like this - beyond overall search volume, what other data might be important in identifying the cause of this. In short, I'm looking for some logic/structure for how I investigate this, using Moz tools and reports.

Thanks.

Mark

omE1VPc

Any thoughts on my question about availability, Russ? I understand you may not be able to offer a precise date but I have no idea whether we're looking at a month, 6 months, a year, etc. Just some sort of guideline would be helpful. For the particular research workflow I have in mind this is really important to me. I have signed for a trial account for AHrefs this morning and they seem to have very good coverage here. But, having been with Moz for a long time, my very strong preference would be to stay with Moz - but something as little more than "one day..." would be helpful

Thanks.

Mark

I see this is being endorsed as an answer to my question but until I know WHEN then I really don't have an answer.

Yeah, I'm kidding. But, seriously, "it's coming" is great and I am excited to hear that, but there's a ton of stuff that is on it's way and, until then, it's just vaporware.

I am in the slightly tricky situation that I need to look at long tail keywords pretty soon, so having at least some ballpark for when we might see this start appearing in Moz, at whatever level, would be cool.

Thanks again.

Mark

Wow - it seems my timing is indeed pretty good. Last week I posted a question on a fitness product site about a missing feature, blissfully unaware that it was about to ship. Six hours later. Beat that... Waiting...

Seriously, this is excellent. I am actively researching the solutions in this area so I guess my next question is reasonably obvious - when will this be available in some form through Moz (and is there a beta)?

Thanks again.

Mark

I recently joined a community related to Internet Marketing and I am finding it very useful. One of the topics that comes up a great deal is what tools folks use to investigate content gaps. I've been a Moz subscriber for quite some time and understand that, compared to many of its competitors, it's a high end solution. So, feel pretty well positioned there.

However, one thing that appears to be lacking is what might be called a long tail discovery tool. I am well aware of Google's Keyword Planner and other free stuff (the problem there being... so are my competitors :)). But there are a number of tools out there - SEMRush, Long Tail Premium, Authority Metrics and so on - that focus very directly on that, with "scores" for keyword competitiveness that at least help narrow down to a possible set, using filtering.

While I accept that one might argue the merits - or otherwise - of each of these specific tools, it causes me to wonder why Moz doesn't have such a capability. I see some very successful web site owners describing their processes in the aforementioned community and long tail analysis is a key part of that.

It seems that Moz has a richness of features, including around keyword ranking. But it's not clear to me why it doesn't support long tail keyword discovery "directly" (as a specific feature). I really don't want to pay for another subscription

Thoughts?

Thanks.

Mark

It does, Tom. Thank you.

Mark

Thank you, Ifemidayo. Yes, that's my conclusion too, though I'd love to be proven wrong There are a number of competitive sites (Moz, SEMRush. SearchMetrics, Traffic Analysis App and so on) that provide this so it is a little surprising Moz doesn't offer this analysis.

Like you, I'd love to hear from Moz folks on this.

Thanks.

Mark

Thank you, Thomas. Yes, that's pretty much in line with my thinking. I have used Moz for quite some time and it is indeed very rich in functionality. It's just a shame that - for this particular scenario (finding the keywords that result in traffic to competitive sites) - there doesn't seem a way to tackle this with Moz.

I was hoping there was some functionality or some combination of features that would allow me to get to this on Moz. It speaks to the breadth of the Moz feature set that I was initially assuming it was there somewhere but I just hadn't located it yet.

Thanks again.

Mark

A key part of my content creation decision is identifying keywords that have been successful for competitors. Ahrefs has a feature that allows me to see the keywords that have resulted in organic traffic for a particular page. So, for example, I can identify CompanyA as a competitor, find their top pages and, for each one, find the keywords that drove the most traffic to each of those pages. The information reported includes the monthly search volumes associated with each keyword, plus the number of visits driven to the page based on that keyword.

That is very interesting to me. I am a long time Moz subscriber and want to find a way to do this here. Is there any way to do this with Moz features? I can't justify both a Moz and AHrefs subscription so I have a tough choice to make if Moz doesnt' support this. I'm hoping I am just missing that here.

Thanks.

Mark

So far so good - every URL I try to my site is now redirected to the www subdomain and, to date, I am unaware of any side effects. I will keep monitoring but all looks good at this point.

Thanks again to everyone who helped with this.

Mark

Yep, totally agree with the thinking and prefer that approach. I researched the non-www (blank subdomain) solution a while back and the rewrite rule I used was suggested in a number of places. But your suggest seems much more robust, providing there's no gotcha (I know of none).

I implemented this and it seems to be working (after a bit of a detour because of caching :)). So, I think I am good and will monitor.

A big thanks to you and Travis. I very much appreciate the prompt responses.

Mark

Here's the only (remotely) relevant entry I have in my .htaccess file (replacing my actual domain with <domain>):</domain>

RewriteEngine On
 RewriteCond %{HTTP_HOST} ^<domain>.com$
 RewriteRule (.*) http://www.<domain>.com/$1 [R=301,L]</domain></domain>

The intent of this is to redirect all URLs of the form http://<domain>.com/page</domain> to http://www.<domain>.com/page.</domain> I implemented that a while back and it seems to be working just fine.

I can't find any reference to the blog subdomain. I suspect what I need to do is implement something like this:

RewriteCond %{HTTP_HOST} ^blog.<domain>.com$
 RewriteRule (.*) http://www.<domain>.com/$1 [R=301,L]</domain></domain>

By the way, despite appearances I am "the guy" for this site. I've completely installed/configured it from scratch so anything broken is my fault  I have the smarts to backup, test, verify, restore etc so I can make changes to .htaccess myself. I just don't have significant experience of the .htaccess file itself so just need to be cautious as I go. But it's all me

Dare I suggest that what I am trying to achieve here (redirect from the blog subdomain to the www subdomain) shouldn't be too tricky?

Thanks.

Mark

Thanks again, Alan.

I am thinking that what I probably need is to 301 redirect all URLs of the form http://blog.<domain>.com/page</domain> to http://www.<domain>.com/page</domain>, yes? In short I don't want to use the blog subdomain anywhere so, to avoid SEO inefficiencies, I should presumably redirect as above so that everything ends up at the www subdomain.

I think

Thanks.

Mark

Travis,

I want to be really sure I understand what you are saying so let me clarify.

I do have a blog DNS entry, referring to the server (just like I have www, to the same IP address).  In a "clean" installation, any page being requested to that server, SHOULD redirect to the www subdomain because I have www.<domain>.com</domain> specified in General \ Settings. Is that accurate so far?

However, "something" (probably in .htaccess) is kicking in that allows URLs with the blog subdomain to resolve (no redirection), regardless of what is in General \ Settings, yes?

Thanks.

Mark

Thanks Alan. I would like to verify one aspect of this.

Somehow, Moz/Google located the blog.<domain>.com pages. I honestly don't know why but I assume that if these pages have actual resolved all along, as they do now, then I could potentially have external URLs pointing to them, yes? If so, then if I just follow the suggestion you have made then won't I continue to have duplicate content issues? I fear I am missing something from your suggestion :)</domain>

PS: I just realized what you perhaps meant. I have generally seen references to "non-www" to mean URLs of the form http://<domain>.com</domain> ie. with no subdomain. But I am guessing you mean "non-www" in the broader sense - absolutely ANYTHING that doesn't have the www subdomain should redirect there. That's presumably what you had in mind, yes?

Thanks again.

Mark

We are using the Redirection plugin.

https://wordpress.org/plugins/redirection/

However, everything I have entered is at the page level (redirect page1 to page2). I don't know if the Redirection plugin even supports subdomain redirection. I am checking that now (a quick scan of the support page for that plugin finds every question related to subdomains unanswered :)).

It seems the working assumption is that nothing changed and that Moz/Google just found this for the first time, which I hadn't considered.

Thanks.

Mark

Thank you Travis. I have never actually edited the .htaccess file directly (not even 100% sure where it is :)) but I guess what you suggest could be put there through other means (plugins, etc).

Just to make sure I understand, though, what I actually want is a 301 redirect for all pages under the blog subdomain to the www domain. I'm not overly familiar with the .htaccess syntax for redirects (but can research) - but if you happen to have the syntax for that I'd be appreciative

Thanks again.

Mark

Thank you Leslie. Yes, I can browse to the blog... pages and they remain that way in the browsers URL field. So, the thinking is that this has been around all along and Moz/Google just found it? Interesting.

So, any idea on how I would fix this? I do use the Redirection plugin (and have been entering quite a few redirects recently) but as far as I can tell that plugin allows me to redirect between pages, rather than redirect subdomains i.e. I can redirect http://www.<domain>.com/page1</domain> to http://www.<domain>.com/page2</domain> but not www -> something else.

Thank you again, Leslie.

I should add that I do have a DNS record for the blog subdomain. However, that has been in place all along, including the last few weeks where these duplicate content errors didn't exist. Given the settings in Settings \ General I would expect these to be redirected to http://www.<domain>.com</domain>, which seems to be what was happening up until this weeks report.

Over the last few weeks I have been focused on fixing high and medium priority issues, as reported by the Moz crawler, after a recent transition to WordPress. I've made great progress, getting the high priority issues down from several hundred (various reasons, but many duplicates for things like non-www and www versions) to just five last week.

And then there's this weeks report.

For reasons I can't fathom, I am suddenly getting hundreds of duplicate content pages of the form http://blog.<domain>.com</domain> (being duplicates with the http://www.<domain>.com</domain> versions). I'm really unclear on why these suddenly appeared.

I host my own WordPress site ie WordPress.org stuff. In Options / General everything refers to http://www.<domain>.com</domain> and has done for a number of weeks. I have no idea why the blog versions of the pages have suddenly appeared.

FWIW, the non-www version of my pages still redirect to the www version, as I would expect.

I'm obviously pretty concerned by this so any pointers greatly appreciated.

Thanks.

Mark

Thank you, Dirk and Chris. Your comments are very much appreciated.

What I decided to do was basically redirect conditionally. What I mean by that is I did the following:

• Exported the 18,000+ 404s over the last week or so
• Created an Excel pivot table to count each URL
• Ordered by number of hits for each URL
• Ignored all URLs that have generated 10 or less 404s in the last week (this gets rid of thousands)
• Filtered out stuff that related to my old CMS (DNN) - these are clearly not relevant anymore. I could do this by pattern matching a few strings (/portals, etc) for DNN-specific URLs.
• Of the remainder, manually scanned for ad-hoc stuff that I know I won't need (old content, images, etc)

That little process got me down from 6,200+ URLs (accounting for the 18,000+ 404s) to a little over 200 URLs for which I will implement redirects. I'm starting at the top of the list (URLs that have generated the most number of 404s in the last week). Some have hundreds of 404s associated with them and there's a clear target URL, so it's good to do these.

I will work through these manually (takes a while :)) but at the end of this I think I will have covered the most important 404s. I'd fully expect my 404s over the next week to be much lower

Thanks again.

Mark

I have recently moved to WordPress (from DNN) and I am very pleased with the change. As part of this switch I took the opportunity to totally revamp my posts, in terms of their URLs, titles, content, etc. I also put 301 redirects in place from the old to the new URLs.

Despite the redirects, logs suggest I have tons of 404s to my site. My site has been around for 20 years this year so I am not particularly surprised by this. Many of these broken URLs are one-offs (and I have no idea why they come in :)) and I am not worried about them. But over the years I have apparently had various versions of the same content, with different types of URL structure. By way of example, if I look at .aspx requests with "human readable" URL (where the URL is obviously an article/post) I have almost 2,500 such requests generating 404s in the last week or so.

My question is how much effort should I put into creating 301 redirects for these old ULRs? I see the recent changes as something of a fresh start so part of me wants to just ignore them and let search engines figure out the new URLs naturally (which seems to be happening). But I also wonder about how much SEO magic I am throwing away here - is it worth spending time reviewing these old URLs, identifying the new URL and implementing a redirect? That's a very time consuming process but I can make it happen if there's real value in doing so.

On a related question, once a search engine detects a 404 is it "done" with that 404 for good? For example, if Google tries to access a URL and finds a 404, does it just wipe the URL and never try again? If so, that presumably means - for that search engine - there's no point in the redirect being in place. It's too late at that stage, right?

Thanks.

Mark

Thank you for the response, Linda. So, this is a slightly tricky one because I don't have a specific deadline per se, but also want to build a plan that gets me over to the new server as soon as possible, without falling into a trap of the switchover date just "floating". Let me put it this way.

I have the following "phases" for each of the articles (as reminder, I have around 200 such articles):

1. Create all articles: Using the planned titles, categories and URLs but with no content.
2. Move content across from old site to the new articles. Done with straight cut-and-paste (don't ask about importing - long story :)). This gets the data into WordPress posts as-is, but includes HTML markup from the old CMS, doesn't correctly use styles (some articles look pretty messy) and doesn't have a consistent use of H2 tags (H1 is the title). Most articles look "OK" but a) some are messy but readable for the human eye and b) the lack of H2 tags means there's no structure from an SEO-perspective.
3. Clean up article HTML/structure. Review each article, cleaning up the HTML and ensuring the content still makes sense and reads well. HTML clean-up includes removing HTML relevant to the old CMS and making sure I have article structure through use of H2 tags
4. Review each article for SEO. Will be using the Yoast SEO plugin and making changes recommended. The keywords are already decided (the URLs and titles in step 1 reflect those decisions) so for each article I will be reviewing the rest of the content and making sure it looks acceptable from an SEO perspective,

I am currently done with step 2 (all articles moved across, albeit some looking somewhat untidy and without any document structure). I am starting to work through step 3 now, but this is a time-consuming process.

I guess what this all boils down to is if I switch across will search engines "catch up" later, when I revise the content for structure and SEO changes. The existing site is not good - so, as it stands, search engines don't look on the site kindly.

One option is to just bite the bullet and move across (I'd see benefits from the title and URL changes, with the associated 301 redirects in place) and subsequently do steps 3 and 4. I'd actually like to do that but ONLY if I can be confident the search engines will end up in the same place as they would if I just waited till step 4 is done.

Another option is to finish step 3, move to the new server and then start updating articles for SEO (step 4).

Thanks.

Mark

I am in the midst of moving my site to a new platform. As part of that I am reviewing each and every article for SEO - titles, URLs, content, formatting/structure, etc, etc. I have about 200 articles to move across and my eventual plan is to look at each article and update for these factors.

I have all the old content moved across to the new server as-is (the old server is still the one to which my domain's DNS records point). At a high level I have two choice:

1. Point DNS to the new server, which will expose the same content (which isn't particularly SEO-friendly) and then work through each article, fixing the various elements to make them more user friendly.
2. Go through each article, fixing content, structure, etc and THEN update DNS to point to the new server.

Obviously the second option adds time before I can switch across. I'd estimate it will take me a few weeks to get through the articles. Option 1 allows me to switch pretty soon and then start going through the articles and updating them.

An important point here is the new articles already have new (SEO-friendly) URLs and titles on the new server. I have 301 redirections in place pointing from the old to new URLs. So, it's "only" the content of each article that will be changing on the new server, rather than the URLs, etc.

So, I'd be interested in any suggestions on the best approach - move across to the new server now and then fix content or wait till all the content is done and then switch to the new server.

Thanks.

Mark

Donna, thank you so much for the response and my apologies for the delayed response (I read as you replied but forgot to thank you). I believe I am on the right track and your responses give me confidence that the changes make sense.

Thank you again.

Mark

I am in the midst of a major redesign of my site, including revamping existing articles . I have a couple of hundred articles and I am reviewing all aspects of these articles, including titles, URLs, content, etc. I am putting together a process as I move each article across to the new site and have SEO very much in mind. I'd appreciate any feedback on this.

First off, let me be clear that I consider the quality of the content paramount. Anything suggested below is considered "supporting" (that content) from an SEO perspective. But, since I am moving this content across, I may as well take the opportunity to clean things up.

The existing articles don't have particularly good SEO-related attributes, in terms of their titles, URLs, use of keywords and so on. So, I plan to do the following for each article. For illustrative purposes (our site serves the wedding industry), I will use an article about how to involve children at a wedding. Questionsunder each bullet.

1. Use the "Keyword Difficulty" feature on Moz Pro to research a specific keyword for each article. In the example case I used "involving children in our wedding".

• Honestly, I am not really sure what to do with this feature I've read everything from "focus on the long tail" to "don't fear highly competitive keywords". So, my current thinking is merely to use it as interesting information for they keyword I choose but not actually make any specific decisions from that ie. make sure the keyword is relevant to the article as the first priority and use the tool to check out search volume.
• Not sure what I should read into a zero for recent Bing searches. Is that really an important factor? I'm assuming the Google information is not available from Google (it would be displayed here otherwise, I'm guessing)

3. Use a title that uses these keywords. In this case, I simply went with "Involving children in our wedding".
4. Same for URL - /wedding-guests/involving-children-in-our-wedding

• If I have a reasonable, short and human-friendly term like this (I can do this with virtually every article quite easily), is there any reason why the URL and the title should not be the same? In short, the title and URL are both a relatively concise "mini-sentence"

6. Make sure the meta description of the article is easy-to-read (for humans) and uses the keyword (sentence)
7. Make sure that the theme (we are moving to WordPress) uses H1 for the page header/title and H2 for sections within the document
8. Implement 301 redirects from the old URL (old site) to the new URL

• This seems like a pretty obvious approach for articles where the URL has changed (which will be most of them). But what do I do with articles that I am going to remove. Should I redirect (301) to a related article (so at least the visitor ends up on a page that is generally relevant) or just let this "fall through" as a non-existent page (401)?

As I say, I have 200+ articles to go through I want to make sure I am taking this advantage to clean things up.

Anything leaping out as missing/problematic?

Thanks in advance

Mark

This absolutely helps - a lot Thank you, Donna.

So, I am narrowing in on the following:

• Navigation (across the site) links to four or so "topic" pages, which are flagged as categories in WordPress. These would be something like "Planning", "Your Wedding Day", "Love and Marriage", etc. They are the "big buckets"

• I have tags like cakes, dresses, rings, guests, etc. Note: I didn't understand your comment that tags should be mutually exclusive. If I have an article about saving money on flowers, why would I not tag that with both "budgeting" and "flowers"?

• Articles

• All articles belong to one and only one category, as implied by their URL structure. This creates some challenges because some articles may have some relevance to more than one category but I think I can live with this

• URLs would be of the form <category>. So, for example \your-wedding-day\choosing-your-wedding-ring</category>

• I can flag articles with one or more tags

• For each of category

• The term I expose on the page (visually) will be "human friendly". What I mean by that is I will use a natural terminology that reads well and is NOT focused on SEO-specifically. So, the navigation menu might simply be "Planning" (as opposed to "Wedding Planning", given that the whole site is about wedding)

• The URL to the category page will be SEO-friendly (so, \wedding-planning, not just \planning)

• The category page will be SEO "aware" (decent title, keyword focused, etc) but also a darned good read for the visitors (quality of content being a factor in SEO anyway)

• The thinking here is that the URL relays the essence of the page for URL but I don't need to clutter the visual expression of this (what the visitor sees) by prepending "Wedding..." to everything.

Am I making sense?

Thank you again.

Mark

Thank you, Donna. So, I've been doing some more thinking about this and it's a difficult balance. I have two ways to look at the whole issue.

• Through the aforementioned "big buckets", such as Planning, Looking Good, Your Wedding Day, etc. I see 4, maybe 5, of these.
• Through the "topics" (cakes, dresses, etc). We have quite a number of those.

The first one is best suited as the second tier ("folders" under the root) numerically - there are, say, 4 of them. However, they are pretty broad and I don't know how successful we'd be in what are very competitive SEO areas (wedding planning, for example, is right up there). Also, apart from the article/post lists (which would be featured in those pages), I am not sure how often they would change.

The second ones are more specific, but are numerically probably too broad for the second level. But the

So, still not sure how to structure this. But one thing I would like to clarify is that if I go with the former (big, broad areas) then having them as second-level entities is no worse, presumably, than just having /blog at the second level, right? It would seem to me that /blog has very little SEO value, whereas even something broad like /wedding-planning has at least some value, right?

Thanks.

Mark

OK, here is something else that is on my mind. We're starting to get into IA at this point and comments are very much appreciated.

So, as I have mentioned I have a number of topics and all of my articles will be associated with one or more of those topics. I plan to use WordPress categories (and maybe tags) to represent those, with the ongoing discussion about URL hierarchy on my mind.

I see the categories as the "meat and potatoes" of how I structure my content. (Since the cat is out the bag) This means cakes, dresses, flowers, etc.

But there's another way to look at the wedding - more along the lines of Planning, Looking Good (dress, makeup, etc), Love and Marriage, etc. These are additional buckets through which a bride might view her wedding.

I'm trying to get my head around whether these concepts are superfluous and/or unnecessary. My worry is that the categories are too granular - though still meaningful - and don't present a simple "roadmap" to the visitor.

Anyway, the reason I am asking is that there may potentially be more than one way to present all this. I like the idea of the "big buckets" above because that first impression is important (I'm taking about the human response here, not search engines) and a list of 15-20 "topics" can be overwhelming. Many of the web sites I used have a first impression that is simple, direct and easy to "consume" - a "1, 2, 3" step process, etc.

I don't know how that maps to SEO-related factors though.

Still chewing on this.

Mark

Thank you. Ryan. I had seen articles like this. It's tempting to see an article that is so old as outdated - but there's also the idea that there are some basic, common sense "primitives" that are just as true today That's a good list and I appreciate the pointer.

So, the flatting is something I have convinced myself is worthwhile. The open issue now is "how flat" See the other comments in this thread.

Thanks again.

Thank you Donna. Yes, I'm finding this issue of "flatness" to be somewhat tricky because I'm getting plenty of alternative opinions - the nature of this while thing, I guess

You are right about the number of articles, which will run into the hundreds. I've been looking at some successful sites and find very little consistency in this area - some are very flat and some have well-defined structure in their URL schemes.

Can you expand a little on why placing the articles under /blog would be better than having them under one of the topics like /wedding-cakes (I am referring specifically to the URL now, not just from where they are linked)?

Your comments have clarified one aspect (and I'm asking here in case there are some contrary thoughts on this). The topic pages may initially be a simple listing (perhaps auto-created by WordPress, based off of categories). For those I could ensure they are not indexed. Eventually, though, I plan to make these more than mere link pages - I'd like to have useful content on them, pertinent to the topic. So, while the page would hopefully be a "darned good read", that content - once done - would be relatively static. If I achieve that goal (the page has useful but static content), would that be the point to have it indexed? If so, that gives me a framework I can work in and one I can move towards over time.

Thanks.

Mark

My site serves a consumer-focused industry that has about 15-20 well recognized categories, which act as a pretty obvious way to segment our content. Each category supports it's own page (with some useful content) and a series of articles relevant to that category. In short, the categories are pretty focal to what we do.

I am moving from DNN to WordPress as my CMS/blog. I am taking the opportunity to review and fix SEO-related issues as I migrate. One such area is my URL structure.

On my existing site (on DNN), I have the following types of pages for each topic:

/ <topic>- this is essentially the landing page for the topic and links to articles</topic>

/<topic>/articles/ <article-name>- topics have 3-15 articles with this URL structure</article-name></topic>

With WordPress, I am considering moving to articles being under the root. So, an article on (making this up) how to make a widget would be under /how-to-make-a-widget, instead of /<widgets>/article/how-to-make-a-widget I will be using WordPress categories to reflect the topics taxonomy, so I can flag my articles using standard WordPress concepts.</widgets>

Anyway, I'm trying to get my head around whether it makes sense to "flatten" my URL structure such that the URLs for each article no longer include the topic (the article page will link to the topic page though).

Thoughts?

Thank you - this is very helpful. I did some basic investigation around this.

The number of these vendor pages that have had at least one hit in the last year is 590. Across these pages I have had a total of 2249 hits. One of the pages (for some reason) accounts for 410 hits, but the vast majority (> 570 of these pages) have less than 20 hits for the entire year. Collectively, these pages have resulted in < 0.5% of our total page hits for the year.

So, they are meaningless in terms of volume of traffic but form a large percentage of our page count.

I am not really sure about the impact of a panda hit (or how to verify) but it does seem that the data above just points to deleting them. Since I think I can 301 redirect with a regular expression to the appropriate "topic page" that seems like the most appropriate approach at this stage.

By the way, I not sure how easy this is to answer, but how would I best assess "a significant rankings boost". Would that manifest itself in remaining pages being ranked better or is this as simple as looking at the domain authority after these pages are removed?

Thanks again.

The vendor links are an important factor for me (and potentially lost in this thread, which is broader in scope). For that reason I opened a separate thread specifically to drill down on that aspect:

http://moz.com/community/q/should-i-remove-all-vendor-links-link-farm-concerns

Thank you for the prompt response. These pages are there specifically to show lists of vendors (links to their web sites). We do identify the relevant topic from the URL and, from the vendor list page, link to relevant content elsewhere on our site, in a sidebar. A typical page like this would link to 5-10 articles elsewhere on our site.

But the primary content is the list of vendors.

Thanks again.

I have a web site that has been around for a long time. The industry we serve includes many, many small vendors and - back in the day - we decided to allow those vendors to submit their details, including a link to their own web site, for inclusion on our pages. These vendor listings were presented in location (state) pages as well as more granular pages within our industry (we called them "topics). I don't think it's important any more but 100% of the vendors listed were submitted by the vendors themselves, rather than us "hunting down" links for inclusion or automating this in any way.

Some of the vendors (I'd guess maybe 10-15%) link back to us but many of these sites are mom-and-pop sites and would have extremely low authority.

Today the list of vendors is in the thousands (US only). But the database is old and not maintained in any meaningful way. We have many broken links and I believe, rightly or wrongly, we are considered a link farm by the search engines.

The pages on which these vendors are listed use dynamic URLs of the form: \vendors<state>-<topic>. The combination of states and topics means we have hundreds of these pages and they thus form a significant percentage of our pages. And they are garbage So, not good.</topic></state>

We understand that this model is broken. Our plan is to simply remove these pages (with the list of vendors) from our site. That's a simple fix but I want to be sure we're not doing anything wring here, from an SEO perspective.

• Is this as simple as that - just removing these page?
• How much effort should I put into redirecting (301) these removed URLs? For example, I could spend effort making sure that \vendors\California- <topic>(and for all states) goes to a general "topic" page (which still has relevance, but won't have any vendors listed)</topic>
• I know there is no distinct answer to this, but what expectation should I have about the impact of removing these pages? Would the removal of a large percentage of garbage pages (leaving much better content) be expected to be a major factor in SEO?

Anyway, before I go down this path I thought I'd check here in case I miss something.

Thoughts?

Monica,

Thank you so much for your response, which is greatly appreciated. As you might imagine, I have a few more questions.

Can I assume that when you indicate that the old URLs are worth retaining that use of 301 redirects falls into that? Or, instead, are you suggesting that I should actually retain the old URLs (no redirect)?

The content issue is a little more unclear to me. It is true that I get a small percentage (about 33%) from those other pages. But that still adds up to 150,000 page views. Not a ton of activity, admittedly, but also non-zero

My (tentative) conclusion at this point is to port the existing content as-is, but map to a better URL structure, redirecting the old URLs to the new ones (301). Then, as I start creating new, more valuable content (with completely new URLs) I will have pages that effectively supercede the older content, and I'd redirect the older URLs to the new ones. At that point I have new content and a new URL structure, so it's something of a phased approach but doesn't necessitate throwing away all my current content. Anyway, that approach is on my mind.

The SEO story around the vendor pages is not a good one Here's how it works. Each vendor can submit their details through a form and select a topic and a location (state). Once approved, that vendor would show on a topic page and a separate location page. We don't have pages where we don't have data (so, if Nebraska doesn't have any vendors we don't artificially create an empty page) - but with thousands of vendors we still have thousands of pages. So, numerically, we are probably well over 50% of these "trash" vendor pages. So, we are presumably in the "world of trouble" right now

I have some questions related to these pages:

• Is there a particular metric or metrics that might "prove" that these pages are doing more damage than good? Does Moz expose anything that readily expresses a page like this being assessed poorly by Google?
• Given that I am almost certain that the pages are "toxic" (but would like to know for sure - see above), would I be best served by simply deleting them (this is easy to do) and then presumably redirect them to the home page since there will be no obvious replacement content. I basically want them gone!
• If I do delete them, over time would the "penalty" be repaired, in terms of SEO

I have a lot of questions, don't I Seriously, this is really fascinating to me and I'm eager to understand this before I make any changes. In short, I need a plan.

Thanks again

I have some tough decisions to make about a web site I run. The site has seen around for 20 years (September 1995, to be precise, is the date listed against the domain). Over the years, the effort I've expanded on the site has come and gone, but I am about to throw a lot of time and effort back into it.

The majority of the content on the site is pretty dated, isn't tremendously useful to the audience (since it's pretty old) and the site design and URL architecture isn't particularly SEO-friendly.

In addition, I have a database of thousands vendors (for the specific industry this site serves). I don't know if it's a factor any more but 100% of the links there have been populated by the vendors themselves specifically requesting inclusion (through a form we expose on the site). When the request is approved, the vendor link shows up on the appropriate pages for location (state) and segment of the industry. Though the links are all "opt-in" from vendors (we've never one added or imported any ourselves), I am sure this all looks like a terrible link farm to Google! And some vendors have asked us to remove their link for that reason

One final (very important) point. We have a relationship with a nationwide brand and have four very specific pages related to that brand on our site. Those pages are essential - they are by far the most visited pages and drive virtually all our revenue. The pages were put together with SEO in mind and the look and feel is very different to the rest of the site. The result is, effectively, a site-within-a-site. I need to carefully protect the performance of these pages.

To put some rough numbers on this, the site had 475,000 page views over the last year, with about 320,000 of those being to these four pages (by the way, for the rest of the content "something happened" around May 20th of last year - traffic almost doubled overnight - even though there were no changes to our site).

We have a Facebook presence and have put a little effort into that recently (increasing fans from about 10,000 last August to nearly 24,000 today, with a net gain of about 2,500 per month currently). I don't have any sense of whether that is a meaningful resource in the big picture.

So, that's the background. I want to totally revamp the broader site - much improved design, intentional SEO decisions, far better, current and active content, active social media presence and so on. I am also moving from one CMS to another (the target CMS / Blog platform being WordPress).

Part of me wants to do the following:

• Come up with a better plan for SEO and basically just throw out the old stuff and start again, with the exception of the four vendor pages I mentioned
• Implement redirection of the old URLs to new content (301s)
• Just stop exposing the vendor pages (on the basis that many of the links are old/broken and I'm really not getting any benefit from them)
• Leave the four important pages exactly as they are (URL and content-wise)

I am happy to rebuild the content afresh because I have a new plan around that for which I have some confidence. But I have some important questions.

• If I go with the approach above, is there any  value from the old content / URLs that is worth retaining?
• How sure can I be there is no indirect negative effect on the four important pages? I really need to protect those pages
• Is throwing away the vendor links simply all good - or could there be some hidden negative I need to know about (given many of the links are broken and go to crappy/small web sites, I'm hoping this is just a simple decision to make)

And one more uber-question. I want to take a performance baseline so that I can see where I started as I start making changes and measure performance over time. Beyond the obvious metrics like number of visitors, time per page, page views per visit, etc what metrics would be important to collect from the outset?

I am just at the start of this project and it is very important to me. Given the longevity of the site, I don't know if there is much worth retaining for that reason, even if the content changes radically. At a high level I'm trying to decide what questions I need to answer before I set off on this path.

Any suggestions would be very much appreciated.

Thanks.

Thank you for the response. I should have been more clear. It is the weekly SEOMoz crawl that is showing such inconsistent behavior, not Google. Sorry I wasn't more clear.

We have very few (if any) broken links, errors, etc.

Thanks.

Mark

I am seeing huge fluctuations in the number of pages discovered the crawl each week. Some weeks the crawl discovers > 10,000 pages and other weeks I am seeing 4-500.

So, this week for example I was hoping to see some changes reflected for warnings from last weeks report (which discovered > 10,000 pages). However, the entire crawl this week was 448 pages.

The number of pages discovered each week seems to go back and forth between these two extremes. The more accurate count would be nearer the 10,000 mark than the 400 range.

Thanks.

Mark

Thank you for the response, Erica. I am aware of the Permalinks settings but, as far as I can tell, these only help with the structure of the initial part of the URL (before the ?). The issue I am seeing - as reported by SEOMoz - are hundreds/thousands of URLs with multiple querystring parameters after the ?. See the example in my original post for an illustration.

Thanks again.

Mark

The problem is I have no idea what these URLs are, whether they are necessary, where I should redirect, etc. I'm looking to understand their purpose and whether there is any way to determine whether WordPress generates them.

What concerns me most is that this is a relatively new blog (just a couple of months old) with a little over 40 actual posts. This is turning into thousands of URLs. I'm struggling to believe that all WordPress sites tolerate this type of "URL noise" so eager to learn what others have done.

Thanks.

Mark

I am hosting WordPress on IIS 7.0. I have a virtual server from the cloud (GoGrid) so while not technically my own I have full and complete control over the virtual machine, so no constraints there (full admin rights, can reboot whenever I like, etc).

What's on your mind (what does that change)?

Thanks.

Mark

Thank you Alan. Ironically, I use a CMS for the majority of my web site pages (except the blog) that is also very ASP.Net centric, namely DotNetNuke. I am very happy with it. I use WordPress just for the blog component.

There are many reasons I went went with WordPress but - although I am an "amateur programmer" (OK in C#, ASP.Net) I much prefer the simplicity and richness available with such a popular product. For example, it installed in 10 mins and within an hour I had the blog on my site, had configured permissions for a bunch of bloggers we have and so on. I also just grabbed a theme I liked and it "just worked". All that without a single line of code.

That said, this particular SEO issue is a problem for me. I was told by numerous folks here that switching to self-hosted WordPress was the way to go from an SEO perspective. I made that change, implemented an SEO-focused plug in (didn't need to develop anything myself :)) and then found what I agree is a "mess". However, the very fact that WordPress is a mature and massively used platform is what gives me confidence that there's a solution here. Hundreds of thousands of web sites use it and I'm pretty sure this sort of issue isn't just "accepted" by all those web sites.

Anyway, those are some of the reasons I use WordPress but my real question is how to resolve them while still using WordPress.

Thanks again.

Mark

As proposed by a number of people here, I have moved from WordPress.com to a self-hosted WordPress blog. I have also installed the SEO All in One plugin. This has been up and running for a month or so.

My problem is that it is generated many (thousands) of warnings through my PRO Dashboard for Crawl Diagnostics. Specifically, I have a huge number of "Overly-dynamic URL" warnings. A typical URL is as folows:

http://www.wednet.com/blog/2011/10/07/do-us-a-favor-dont/?utm_source=rss&utm_medium=rss&utm_campaign=do-us-a-favor-dont

This has three querystring parameters, all generated by WordPress automatically.

Here's another significant issue. With the SEO All In One plug in I can control the SEO-related parameters for each post (title, meta description, etc). However, WordPress generates a ton of virtual URLs which I can't (as far as I know) directly control. For example, the following page is a category page with all the posts for a single category. This is generating warnings because the meta description is missing. However, I do not know how to control such parameters since the page is automatically generated.

http://www.wednet.com/blog/category/ceremony/

These type of warnings dominate the stats I have through my dashboard. How can I resolve these?

Thanks.

Mark

Thank you for the response. I do not have a robots.txt file. I'm not following how the Google crawl rate effects this (but would love to learn why). I thought that is only used to define how often Google itself crawls my site and has nothing directly to do with SEOMoz. Also, it is not how often the SEOMoz crawl takes (it starts on time each Saturday) but, rather, how long it takes to complete. For two of the last three weeks it has taken almost 48 hours from start to finish.

Thanks again.

Mark

I have had my site registered for a long time (since January 1995) and in that time we have built a good number of incoming links. We have a vendor database that we expose as a service to our visitors, which numbers around 5,500. These sites vary dramatically from low-end, Mom-and-Pop type web sites (some ugly in the extreme :)) to nationwide, established vendors.

Back in the day, we had the basic tactic to request a link to our site if the vendor wanted to be listed in our vendor database. We stopped that practice years ago but still have many sites linking to us.

The quality of some of these sites is very poor. I want to come up with a strategy for dealing with these. To that end, some questions:

• How "costly", from an SEO perspective, is a poor quality site that links to our site?
• What metric(s) should be used to assess the quality of sites linking to us?
• If possible, for the aforementioned metric is there a "bar" we might try to hit? For example, would it be useful to request removal of links where <metric>is less than x? What is x?</metric>
• Given that we have thousands to assess, is there any report I can create to identify these sites?
• Is it generally preferred to have vendors link simply to our home page or is it more effectice to have them link to particular pages on our site (each vendor can generally be associated with a "topic" on our site).

In short, I am willing to go through this process if there is real value in it.

Thanks.

Mark

Once again the weekly crawl on my site is very slow. I have around 441 pages in the crawl and this has been running for over 12 hours. This last happened two weeks ago (ran for over 48 hours). Last week's crawl was much quicker (not sure exactly how long but guessing an hour or so).

Is this a known issue and is there anything that can be done to unblock it? Weekends are the best time for me to assess and respond to changes I have made to my site so having this (small) crawl take most of the weekend is really quite problematic.

Thanks.

Mark

Still running and approaching two days now

My first month trial is up today. This has got me thinking about whether to continue. 40+ hours just seems too much or a small web site such as mine.

Mark