Hi Manuel,

This blog post Web Design Company Fail #1 will give you an idea of some of the things to watch out for when redesigning sites.

One trap with Wordpress sites is inadvertently using a template that has built-in SEO problems, so do some good research on the template you select to make sure that there are no known issues identified.

The other thing is to make sure that you plan right from the start to eliminate the standard duplicate content problems generated by default in Wordpress. Using a good SEO plugin for Wordpress is a good idea for this reason. I find that Yoast SEO for Wordpress is very helpful.

Hope that helps,

Sha

Yes, a full crawl will generally be completed once a week.

You can see the date when the current crawl is due to complete at the bottom right of the crawl diagnostics overview for your campaign.

This is what it looks like

Last Crawl Completed: Nov. 29th, 2011 Next Crawl Starts: Dec. 6th, 2011

Hope that helps,

Sha

No probs...the Rewrite Rule goes in the .htaccess file.

Hope your campaign is successful

Sha

Hi Gary,

Yes, the experience for the user is exactly the same. Any request for example.com/tv will tell the server to load example.com/product.

This will solve your problem without any other issues.

Sha

Hi Gary,

A 301 Redirect should only be used if the change of URL is unlikely to ever be changed. One of the reasons for this is that a 301 Redirect also signals to Search engines that the originating URL should be de-indexed and replaced with the target URL.

Since you could conceivably wish to use the same advertised URL at some time in the future, but send it to a different product, this is likely to be problematic.

My suggestion would be to use a simple URL Rewrite Rule instead of a Redirect. URL Rewites basically tell the server to load the target URL when a request is received for the originating URL.

A URL Rewrite has no effect on Search engines and can be removed or changed at any time with no effect at all.

Hope that helps,

Sha

Hi Miranda,

Yes, the initial "fast" crawl is limited to 250 pages. It gives you a quick start opportunity for working on your campaign while waiting for a full crawl to complete.

This is an improvement that was added a few months ago. Before that time you had to wait a week for the full crawl if your site was large.

Hope that helps,

Sha

Hi,

If these are pages that you "don't want to be ranked" then you should add meta "noindex" tag to them immediately so that they are removed from the index and will not be added with new crawls.

As evidenced by your own question, they are pages you don't care about. That means you have no reason to make them great pages. This said, leaving them available for indexing is a "panda slap" waiting to happen.

I would treat the "noindex" as a priority.

Hope that helps,

Sha

Nice solution Kieran!

I use the same method, to compare URL list from Screaming Frog output with URL Found column from my Keyword Ranking tool - of course it doesn't catch all pages that might be indexed.

The intention is not really to get a complete list, more to "draught" out pages that need work.

Hi Kenji,

That would be great - the other thing to look at is developing awareness of your brand through other channels so that people get to know about your organization and come looking for you specfically. It might be worth talking about this with those people you include in your brainstorming.

For example, can you:

• distribute information through schools/universities so that students know about you?
• spread the word through philanthropic organizations to individuals & corporates?
• connect with media/film students at a local college to get short videos, articles, presentations developed for your organization as part of their project work?
• Get some high profile benefactors & students who have been successful to participate in social media, relating their experiences & answering questions?

Hope that helps,

Sha

Hi searchpl,

If you are worried about "freshness" for ecommerce sites, there is one very important thing to do - eliminate wasted effort.

The fact is that what I call the "freshness effect" does not apply to every keyword term. Google appears to be determining whether fresh information is, or is not more relevant according to the individual term. If you manually check SERPs you will see this easily.

So, eliminating wasted effort while working toward providing new and relevant content all comes back to good old fashioned research. The smart approach is to spend some time manually checking SERPs for your "money" keywords. If you see evidence of the "freshness effect" for particular terms, those are the ones you could consider focusing new content development efforts on.

The keyword terms that might be affected will entirely depend upon the types of products in your stores - for example, I know that "weight loss" is a term where the "freshness effect" seems evident in SERPs.

Of course, if you decide to develop new content you should follow the advice already given by EGOL and James on quality and method. Incidentally, I would say the smart thing to do in this situation would be to come up with the type of content that is easy to add on a continuous basis - things like ongoing series, videos, podcasts, and cleverly managed user generated content.

Incidentally, if you listen carefully to information coming out of Bing via Duane Forrester, you may notice that Google is not the only Search engine that takes notice of freshness

Hope that helps,

Sha

Also: Don't stress too much if you are using automated feeds to update your product offerings on a daily basis ... you may already be providing fresh content if products are frequently added. The challenge then is to ensure that quality is up to scratch

Hi vittana_seo

Given the unique nature of the service you are providing and the specific "customer" groups you wish to target, I would go right to the source.

Talk to/set  up brainstorming sessions with representatives of each potential group and ask them how they would describe what you do (terms they might type into a Search Engine etc):

1. past recipients
2. potential recipients who are in the system
3. past contributors - Individuals
4. past contributors - corporate

Hope that helps,

Sha

Hi jassy,

I think the question is basically "is it a problem that most of the external links are pointing to the original page and not directly at the one I prefer (and have redirected to)".

Since the chances of getting external links altered is slim & most of the link value is passed with a 301, then the broad answer would be "no", but the more important task would be to fix the redirect issues you highlighted as quickly as possible.

Sha

Hi Matt,

Search engine crawlers will follow the redirects and pass link value as long as they are 301 (Permanent) Redirects.

Hope that helps,

Sha

Maybe a little like using a 10 pound sledgehammer to crack a peanut shell, but those vector images are always going to be good to look at!

Sha

Hi Noah,

Obviously Rand is the one to answer the first part of your question.

As for the blog post you mentioned, I'm thinking perhaps it was Dr Pete's "Content Marketing for the Talent Impaired"?

Sha

Hi Chukwuemeka,

First a quick note - it is not possible to write a script that does this using html, but by converting your normal html files to PHP, you can achieve your aim for a standard site without having to use a CMS like Wordpress.

This post in the Youmoz Blog Adding "Related Post" Links without a Database should provide everything you need to make it work.

I hope this will help solve your problem ... if there is anything that you don't understand just let me know in the comments or come back to this thread.

I owe my boss a BIG beer gift this Christmas for writing the code that makes this work

Sha

Glad you got it worked out

Don't forget if you need extra help on something you can always use the Private Message system in your profile to contact people direct.

Have a great weekend,

Sha

Hi Matt,

I would agree 100% with Ryan's comments on robots.txt.

In addition to this, it is a notoriously unreliable method for blocking...if the crawler happens to come to your site via an external link to any page other than the home page it will not see robots.txt.

Sha.

Hi Neil,

I would say that probably the perfect place to seek a highly technical answer like that - and have the opportunity to explore the subject if you need to - would be the SEOmoz Engineeering team.

My suggestion would be that you use your Private Question here in Q&A (you get one private question a month as a Pro member) to ask for some help.

An administrator will make sure that your private question is directed to someone on the SEOmoz staff with the specific expertise to help you out.

Of course, with Thanksgiving it may take a little longer than usual, but I would ask now anyway - I'm constantly surprised by Mozzers responding out of hours etc.

Hope that helps,

Sha

Hi Heather,

I think the closest you would get would be to run a Whois lookup for google.com, which will show you a series of IP address ranges that are associated with the domain.

While it will not give you the specific addresses for Analytics, it will also ensure that you are not blocking any of the various googlebots responsible for crawling, whatever job they do

Hope that helps,

Sha

Hi Zsolt,

Did you actually look at the link I gave you with the code that you needed?.

The answer is to stop trying to use Redirectmatch and use the code I gave you in that example:

RewriteEngine on

RewriteBase /score

RewriteCond %{QUERY_STRING} score=
RewriteRule ^/*$ newlocation.html? [R=301,L]

This is the simplest, cleanest, and most reliable solution to your problem.

Sha

Hi again,

We set up an example page for you with working tests and links to example code and zipped version.

Hope that is what you need,

Sha

Hi Zsolt,

Just so we understand exactly what you are asking here ...

What you are wanting to do is permanently redirect any and all URLs containing "thisword" to a single URL on the same domain? Is that accurate?

Sha

Hi Vipin,

Since you have all of the URLs in your robots.txt file (which incidentally is pointless since the pages are no longer there to crawl), I would suggest as follows:

• Create a 301 (Permanent) redirect in your Windows Server's "Internet Services Manager" (since you have moved files to .aspx you will be using a Windows server)
  Since you indicated that you "moved" the pages in question from html to aspx, I would expect that you should be able to redirect to corresponding pages.
  If I were doing this in a LAMP system (not Windows) I use a single line of code to rewrite all .html extensions to .aspx as a 301 Redirect.
  Logically, it should be possible to do the same on a windows server, but since ours is a LAMP Shop I can't offer specifics on that I'm afraid

• Once the redirects have been created, remove all of those URL's from the robots.txt file

Hope that helps,

Sha

Glad to help

Just another thought on using video on-site...if it were possible to grab very short video reviews from actual clients while using your services, that could be gold in terms of converting visitors to your site.

Not entirely sure about the detail of how to make it happen, but I'm imagining a happy, smiling bride and groom shooting a 30 second video snippet while in the back of the limo being driven to the reception..."the limo is gorgeous, our day has been wonderful and Joe our driver even brought barley sugar candy to help settle my butterflies on the way to the church!"  these are the type of videos that will engage your audience and tell the story of your business.

Best of luck with it,

Sha

Hi again,

I would say that both approaches are valid, as long as you are very strategic about targeting your content.

My personal preference is to work first on building better content on-site (I am a member of the "why help someone else when you can help yourself" movement :), but if there is an opportunity to use the domain authority of another site to improve your own site, then that can obviously be of value too (especially if you are trying to build the strength of your own domain).

The key is to target the content you develop for each and NEVER use the same content for both.

When it comes to targeting, I follow two basic rules - articles on other sites should be aimed at an audience seeking "how to" assistance, while on-site content should be the type that will help to "engage" your audience with your company or brand.

For example:

An article on an external directory might be "How Early Should I Book My Wedding Limo", while a new page of on-site content might be "Why You Can Trust Us To Drive Your Daughter To Her Prom"

Hope that helps,

Sha

Hmmm...

Hi echo 1,

My view of Press Releases is quite different. The sole aim of a press release is to provide writers/bloggers/journalists with the key information they need to research and write stories or articles of their own which will promote your site (or your business, products & services).

A Press Release that is likely to be successful in achieving its aim is by necessity quite structured, following a pre-determined format and containing particular facts and information ordered quite specifically to make the prospective writer's job easier. So, by its very nature, a good Press Release is not what I would consider suitable page content for my site. For this reason I would never consider adding releases to my on site content.

Of course, the other natural conclusion from this is that if my Press Release reads like great page content for my site, then it does not follow the basic principles of a well constructed release and should be looked at closely before it is submitted.

If you are submitting to the major online Release sites, they all have resources which will basically teach you the format that you should use for your releases.

Hope that helps,

Sha

That is not going to look good to any search engine. Do you have many other external links?

I'll look forward to hearing how you go.

I've thrown together a quick 10 question survey to gather some information about the effect of the algorithm change. If you would like to be included you can go to Google algorithm change survey.

Sha

Well that is great news Tommo so then the next thing to look at is the content itself.

There is an explanation of the two stated algorithmic changes that relate most closely to your issue in the 2nd & 3rd dot points in this Google Webmaster Blog post.

Remembering that your problem is a drop in traffic, it seems that what Google is pulling from your page text is not moving searchers to click through to your site. So the challenge is to ensure that the text on the page includes the most useful information possible, written in such a way that it motivates people to want to buy the product. If you can get this right, whatever Google pulls from the page, it will be more likely to result in clicks.

This is not an easy thing to do, but if you can achieve this aim, you are likely to not only improve the current problem, but also improve sales conversions.

Again, I would run a test with a small number of pages and see what works.

Hope that helps,

Sha

Hi again Tommo,

Yes, I would say that it is very likely that what you are seeing is the result of what is loosely referred to as a "panda slap". This is why I suggested that I believe you need to look at the content of your pages.

The clue is that Google does not think your Titles accurately reflect what is on your pages and when it serves what it thinks is accurate, less people are clicking through.

If you want to test this, I would choose a small group of my highest value keywords, roll up my sleeves and work on improving the content. When the improvements are made, if rankings and/or traffic improves for those pages, you have your answer - and a clear strategy for fixing the problem

Also, when you say "product pages" my first thought is "are you using product information that is provided by manufacturers?". If the answer is yes, then you are likely to have a serious problem with duplicate content. Dr Pete talked about duplicate content and panda in his recent blog post Duplicate Content in a Post-Panda World.

Hope that helps,

Sha

Hi Chammy,

With no other information on how he came to "lose" it, I would say his only hope is if he still has access to the Google Web Master Tools account, but he would have to act NOW, in the hope that the new domain owner is slow in getting things organized.

The process:

1. Purchase a new domain
2. Upload all content and ensure that site is ready to go as soon as new domain resolves
3. In Google Webmaster Tools, create a Site account for the new domain & verify it
4. In your old Webmaster Tools Site account, notify Google that the site has moved to new domain
5. Add a few pages of completely new content to the site on the new domain
6. In the Webmaster Tools Site account for the new domain, use "Fetch as Googlebot" to fetch each of the new pages and up to the maximum 50 most important pages in the site. (If a page is successfully fetched with this feature, you can manually add it to the index.)
7. Find all existing backlinks possible (using Open Site Explorer, Majestic SEO, link: search operator in Google, Bing & Yahoo) and contact as many of those webmasters as possible, advise of the domain change and ask them to correct their links.

As Alan pointed out, the only way that this will work for him is if the new domain owner is slow in setting up a GWMT account and he still has access to his existing account, so basically, he needs to act immediately and pray a lot

Incidentally, if he has already missed out on the opportunity to tell Google about the change of domain he should still follow the steps above for the new domain and try to get the links corrected if he can to get some quick traction where possible.

Hope that helps,

Sha

Hi Tommo,

Without actually looking at your site, it is pure guesswork, but the key here would be that Google thinks what it is serving in the results page provides the searcher with a better insight into the content of your pages than the Titles and descriptions that you have loaded.

This would indicate that perhaps your Titles & descriptions may be misleading or keyword stuffed.

Since the overall result is that serving what Google thinks accurately reflects your page content has lead to a dramatic loss of traffic, I would say it is time to look very closely at the content on your pages and think about improvements that may make them more interesting and useful for users.

Sorry to bring bad news, but I hope it helps,

Sha

Hi Michael,

I haven't used the plugin myself, but have seen a number of references to it around the traps.

I do remember seeing some information that mentioned it is used by well known internet attorney Mike Young on his website mikeyounglaw dot com. Perhaps you could contact him direct and ask if he can offer any comment.

Hope that helps,

Sha

Hi Ian,

The first place I would look to identify a reason for significant change in places position would be the number of reviews the site has and whether the other sites have gained more reviews.

In the USA, # 1 Listing in Google Places has an average of 33 reviews.

Other Page 1 Listings in Google places have an average of 26 reviews.

If other sites have increased the number of reviews they have and your site has remained static, or not also increased its reviews, this would account for the drop in position.

Hope that helps,

Sha

Hi Michael,

You do not need to make any changes to your .htaccess file. Actually, if you 301 these URLs you will break your search so that it no longer works.

The solution I would use is to go into Google Webmaster Tools and tell Googlebot to ignore the parameters you are concerned about.

In your code, the ? says "here come some parameters" and the & separates those parameters. So, in the case you have quoted, the parameters are a, b, c, d.

Be aware of course, that Roger will still see these URLs as duplicates since he doesn't know about your private conversations with Google   This means that they will still appear in your SEOmoz report, but as long as you make a note of them so you know they can be ignored that shouldn't be a problem.

Hope that helps,

Sha

Hi again,

There are a few really helpful posts you can take a look at that come from Duane Forrester. This one on the Bing Webmasters Blog is about developing great content.

Duane also gives some great Bing specific information in these Whiteboard Fridays from March, June and October.

Hope that helps,

Sha

Hi Zack,

I think most things have been covered, but I would just add:

Start at the beginning. Your title needs to be capable of earning the clickthrough all by itself. The current title is ambiguous - is Austin a city in Texas or a guy with a name that starts with "A".

Consider the difference in impact of a slight wording change. For example "Why is Austin such an angry city?".

Hope that helps,

Sha.

Hi Dominic,

If you want pages to be removed from the index and ensure that they are not crawled again, you do exactly what you did with the others EXCEPT for one detail - the robots meta tag that you use should be "noindex, follow.

If you use "nofollow" you are trapping all incoming link value on that page and preventing it from flowing internally through your site.

Yes, you can add robots meta tags using the Yoast SEO plugin. You will find the features you are looking for in the "Indexation" section. Read the support documentation and on-page instructions carefully and you should be fine.

Hope that helps,

Sha

Hi strasshgoa,

Good advice from Calin - my guess would be that you don't have a redirect in place for that, or that you may have some other canonical issue, perhaps caused by having written the same URL differently in a link. An example of this would be using both www.mysite.com and www.mysite.com/index.html in your code. While both call the same page, they are different URLs and therefore seen by the crawler as duplicate pages.

The easiest way to identify the problem is to click the blue links in the column to the right of the URL that has been identified as having a duplication issue in your Report. The number of URLs that have been identified as duplicates of the page will appear as a link and when you click the number you will see the list of URLs.

There is also a help page for each of the tools in the Pro App which you can access by clicking the tiny blue "? Help" link to the right of the page towards the top (directly opposite the summary link on the left of the page). The help page for Crawl Diagnostics is here.

Hope that helps,

Sha

Hi Alex,

As far as I'm aware the crawl will always come back to the day that the first crawl was scheduled.

However, if you want to find out for sure, the quickest way to get a response would be to email the help team direct - help [at] seomoz.org.

Hope that helps,

Sha

Hi michelleh

The solution given by Dan above is the most reliable method as robots.txt will not block pages that googlebot finds via an external link to the page. Given the reasoning behind your desire to noindex, reliability is extremely important.

Also, you want "noindex, follow" rather than "noindex, nofollow" as the nofollow will trap any link value coming into the pages (from both internal and external links) and stop it from flowing through the site.

Hope that helps,

Sha

Thanks Alan,

I like a little mystery hunt

Hi Jason,

There is obviously something going on with this that is affecting what some crawlers are seeing on your pages.

I ran the Screaming Frog Tool and it shows that the majority of your pages have empty Titles even though I can see that there are Titles loading in the browser.

On checking your code I see that you are using the pragma directive meta element , but it actually appears below the Title element in the code.

Example from your code:

<head>
<title>Are You Socially Awkward? | Branding Blog | The Bullettitle>
**<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />**

So I ran the page through the W3C Markup Validation Service  and it also indicates that it sees no character encoding declaration:

No Character encoding declared at document level

No character encoding information was found within the document, either in an HTML meta element or an XML declaration.

So, I believe the issue here may be related to the fact that the pragma directive should appear as close as possible to the top of the head element ie before the Title element.

The following is from the W3.org documentation on declaring character encoding. You will see that there is specific reference to the fact that the use of the pragma directive is required in the case of XHTML 1.x documents as yours is:

For XHTML syntax, you should, of course, have " />" after the content attribute, rather than just ">".

The encoding of the document is specified just after charset=. In this case the specified encoding is the Unicode encoding, UTF-8.

The pragma directive should be used for pages written in HTML 4.01. It should also be used for XHTML 1.x documents served as HTML, since the HTML parser will not pick up encoding information from the XML declaration.

In HTML5 you can either use this approach for declaring the encoding, or the newly specified meta charset attribute, but not both in the same page. The encoding declaration should also fit within the first 1024 bytes of the document, so you should generally put it immediately after the opening tag of the head element.

Hope that helps,

Sha

Hi again,

If it is queued then you should be fine.

SEOmoz Associates and staff do check questions periodically and would be able to either flag your question with the Help team (Associates don't have access to your account), or deal directly with the problem. This may take some time to happen though, depending on when you post and when they happen to visit.

If it is something that actually needs investigation or action from SEOmoz, then the quickest way to get an answer is to send it straight to the Help team.

Sha

Hi Susanna,

Just wanted to post a quick update on this question since I understand the problem was resolved with the suggestions that we made after looking at the content of your .htaccess file.

The major issue with the original redirect was that the $1 had been omitted from the Rule. So the problem actually was not related to the secure protocol - just a coding error

Example code for the secure to non-secure redirect you needed to implement:

redirect away from https to http

RewriteEngine On
RewriteCond %{SERVER_PORT} ^443$ [NC]
RewriteRule ^(.*)$ http://www.yourdomain.com$1 [R=301,L]

Hope that's helpful for anyone who may be looking for help with the same issue in the future.

Sha

Hi Bethany,

If you are still seeing the message that says the next crawl is due to complete November 1st, then you will need to email the SEOmoz Help team direct and ask them to take a look at it for you - help [at] seomoz.org

Hope that helps,

Sha

Hi Shebin,

The best thing to do is email the Help team direct - help [at] seomoz.org

I had a campaign with a similar issue a while back and the Help team sorted it out for me very quickly.

Hope that helps,

Sha

Perfect!

Glad to hear it's all fixed