Web Site Migration Testing and SEO-QA Automation?

[George.Fanucci]

Hey Mozzers, Are there any good Migration-SEO-QA Tools out there?

Given a prioritized list of URLs and prioritized list of Keywords, is there a tool that can compare basic SEO factors, old URL vs. new URL, and identify all the specific gaps that need to be fixed?

Here is a basic SEO-QA acceptance checklist, for porting any website.

. . . Until the porting work is completed we cannot accept the new website.

Givens:

1. A list of the Top 100 URLs from the old site, prioritized by conversion rates, landing page traffic, and inbound links.

2. A list of the planned 404 - mapped URLs, old to new site, from the porting team.

3. A list of the current Top 200 Keywords, prioritized.

4. A good amount of SEO work has already been done, by several professionals, for the current (old) site.

**How to evaluate if the new site will be acceptable to Google?  Check ON-PAGE SEO Factors... **

**. . . that is, the NEW site must be AS GOOD AS (or better than) the current (old) site,
in the eyes of Google, to preserve the On-Page SEO work already done. **

Criteria:

• URLs ok? :: Is the URL mapping ok, old to new, best web page?
• LINKS ok :: Are all internal LINKS and keyword Anchor Text ported?
• TEXT ok :: On-page content, TEXT and keywords ok?
• TITLE ok :: HTML Title and title keywords ok?
• DESCRIPTION ok :: HTML Meta Description ok?
• H1, H2 ok :: HTML H1, H2 and keywords ok?
• IMG kwds :: HTML IMG and ALT keywords ok?
• URL kwds :: URL - keywords in new URLs ok?

Potential porting defects:

• Keywords in URL missing:
• Keywords in HTML Title missing:
• Keywords in Meta Description missing:
• Any internal LINKS or Link anchor text missing:
• Keywords in Page TEXT missing:
• H1, H2 missing keywords:
• HTML IMG alt-text, IMG file URLs, any missing keywords:

Notes:

• Until the porting work is completed we cannot accept the new site, or set a target date for potential cutover. There are eight (8) data items per URL, and about one hundred (100) URLs to be considered for SEO-QA before going live. We were expecting to cutover before the end of February, at the latest. There is no point in doing full QA acceptance-tests until the porting work is completed.
• QA spot-checks have found far too many defects. About 60% of the landing-page traffic comes via the top 40 URLs. With over 100 URLs to look at, it can take more than a week or two just to do SEO-QA in detail, manually, item-by-item, page-by-page, side-by-side, old vs. new.
• Spot-checks indicate a business disaster would occur unless the porting defects are fixed before going live.

_Any Migration-QA Tools?_Given a prioritized list of URLs and prioritized list of Keywords, is there a tool that can compare basic On-Page SEO factors, old URL vs. new URL, and identify most of the specific gaps that need to be fixed before going live with the new site?

_ *** Edit: Any comments on the SEO criteria, tools, or methods will be appreciated!_

[adammason]

Most cases like these are catered for on an adhoc basis, usually done by a dev team that will migrate the entire database into the new site.

There could well be a market for a porting tool that provides a checklist type system for a large scale site migration. I have seen site migrations go wrong. A lot of revenue can be lost through the plummet of organic traffic from a failed migration.

[George.Fanucci]

Hi Adam, More good points. As you wrote:

• The title, meta and content is copied across so it is blatant to Google that the redirect is in the correct place.

How to efficiently and iteratively check and re-check this work with SEO-Migration-QA Tools? ... For a list of a few hundred URLs .. old ..vs.. new?  (Without a labor-intensive process.)

Surely, this is a common problem and has already been faced, and solved, by many.

If a such a Porting-QA tool does not yet exist, there must be a good market for one?

[adammason]

Hi George

It looks like most of it has been covered here but I thought I would mention a fairly simple way to do this sort of site migration. Firstly I would assume the all title tags, meta descriptions and h1 tags will be the identical to the site transitions. Understandably the URL structure may change slightly.

The number one thing is to make sure that that redirect strategy for the entire site is done correctly. This may mean crawling the entire old site using Screaming Frog to then map out exactly what URL will be redirected to what in the new site. Then your developers can structure this into redirects on the site. Depending on how many pages there are this may need tweaking to make sure there are not too many rules created to redirect the site.

In terms of making sure that each page is setup correctly from an on page point of view, this would be a very large task to accomplish. I am just typing out loud now but you could do the following:

• In Google Analytics export the top 100 (or larger depending traffic coming into your site).
• Export the rankings data that these pages are bringing traffic for organically (this will be difficult now not provided is the majority of organic keyword data but you may have historic data in GA that could help with this).
• Using the highest converting keyword for each page, use a tool such as the Moz onpage grader to fully check the page for any on page issues and an overall grade. You can then run the Moz on page checker on the new dev site and see if the grade is the same. If it is not then make the recommended changes to meet the checklist.

This is by no means an automated system but at least it is running each page using the highest converting keyword on the traffic bringing in the most traffic. This is the best way I can see this happening, it still isn't fully accurate as it is only checking the highest converting keywords but with the page potentially bringing in traffic from so many keywords it is very difficult to automate this fully.

But really the best thing to make sure is:

• The redirect strategy is bulletproof to ensure all backlinks are redirected to the new site.
• The title, meta and content is copied across so it is blatant to Google that the redirect is in the correct place.

I hope this help.

[George.Fanucci]

Hi Ryan, and thanks, again! You do raise good points on Quantitative vs. Qualitative factors.

Yes, the human component is important, but Googlebot+Algo is a mechanical device, and given the list of top 100 URLs and top 200 keywords, prioritized, it should be possible to inspect each page, old .vs. new, using an SEO-QA tool, mechanically. If the porting team invented random garbage for HTML Titles, Meta Descriptions, etc. then the new SERPs would be devastating to the business as soon as the test site goes live.

The new site may LOOK better to the naked eye than the old one, but Google will not treat the new site very kindly in the SERPs. There is always room for improvement but when a site already has had lots of SEO work done, and the porting team omits that from the new site, a disaster looms.

Checking a prioritized list of ON-Page SEO ranking factors for a prioritized list of 100 URLs and prioritized list of 200 keywords should be a fairly mechanical task. A QA tool could do it in a few minutes, while a human may take a few weeks.

**The SEO-QA-Tool could indicate a prioritized list of major SEO gaps and problems to be fixed before going live, for a HUMAN to review and take well-considered action on. **

_Update/Edit :: SEO is never really DONE, there is always room for improvement and things are always changing... even the SEO Ranking Factors. There are currently accepted authoritative opinions that have been published here, and in other places, for SEO Ranking Factors and generally accepted SEO standards. The difference between $4 per hour and $100 per hour SEO work is the quality of the SEO tools being used, and how well those SEO standards are understood, and even the experts often disagree. _

Ref: http://moz.com/search-ranking-factors and http://moz.com/blog/ranking-factors-2013 and http://moz.com/blog/future-of-search-ranking-factors and http://searchengineland.com/seotable

_Update/Edit-2: The key point here being, the SEO features of the OLD site have not been migrated to the NEW site, so the porting team has more work to do. How to QA that work without a huge labor expense? Some basic SEO-Migration-QA tools are needed, based on a sound SEO methodology such as:  _http://searchenginewatch.com/article/2115729/10-Steps-to-a-Successful-SEO-Migration-Strategy

[RyanKent]

Hi George. These items you referred to are not mechanical as you suggest.

Title - How do you determine a "good" title? If your site is Wikipedia, a tool can certainly be used. Wikipedia titles are short and focused on a single keyword. The H1 tag and URL match or closely mirror the title. The first words of the first sentence also use the title.

For sites other than Wikipedia, how do you measure the quality of a title? Tools can share whether or not a keyword is identified in the title. They can share whether a title has more than 70 characters. If you desire optimal results for the site, other factors must be evaluated. Let's look at the title of this Q&A and just think of the first part you shared, "Web Site Migration Testing". Should it be "Web Site" or "Website"? "Testing" or "Tests". What about "Website Migration" or "Best Website Migration Tools". Or "What are the Best Website Migration Tools?" There are dozens of other variants. How will a tool decide upon the optimal title? Will it consider keyword research or existing content on the site? What about competitiveness....and so forth.

The same logic applies to the rest of the criteria you mentioned. For ALT tags the first and foremost consideration I use is "Does the ALT tag accurately represent the image? If a blind person heard the ALT tag read to them, would they be able to accurately describe the image?" As an SEO, I base image naming and content first and foremost on the user experience and then make minor adjustments to improve SEO. Theoretically, you can have a tool recognize images, provide the ALT text and adjust for SEO, but we are a long way off from that vision becoming a live tool in the field we can use.

You can certainly use tools to gather that feedback for specific elements such as title length, does a meta description exist, does an ALT tag exist, etc. As to determining whether those elements are ideally optimized to provide the best results for a particular client, that requires the human element in my experience.

[George.Fanucci]

Hi Ryan, Thanks for taking time to reply. You have some good points, but for the purely MECHANICAL factors such as HTML Title, Meta Description, IMG file names and HTML alt tags, and listing all the internal (and external) links with anchor text — side-by-side, old .vs. new — an automated On-Page SEO QA tool — would surely be very useful to assist anyone porting a website. I am surprised not to find one online as yet.

It's a kind of in-vitro A/B testing. Without such a QA procedure, going live with a test site could be disastrous, if the old site ranks well, the new site may rank very poorly.

We do have a list mapping old-url to new-url, prioritized. We have a list of keywords, prioritized.

Surely there is a niche market for some kind of automated Porting-QA reporting tool?

In a perfect world, there would be an SEO tool to predict the impact of porting before going live. Look before you leap.

[RyanKent]

You can use Moz, WooRank or other tools to review a site, obtain a grade along with other helpful information; however, you cannot depend on these tools to even come close to performing the type of analysis you are seeking. To use an analogy, you can use a template to draw up a will or other legal document, but if it is really important you would always work directly with a lawyer. The same applies for other fields whether it is a medical opinion or SEO advice.

Let's break down one component of your request: "URLs ok? :: Is the URL mapping ok, old to new, best web page?"

What EXACTLY is meant by URLs ok? A full breakdown would be:

1. Protocol. Are pages served as http or https? Which is best for the site?

2. Is the domain name optimal for the site?

3. What is the best URL paths for the site? If you operate a clothing e-commerce site should the primary category be the brand? gender (men vs women)? kids? something else such as clothing type (shirts / pants / suits / etc). That decision cannot be made by any tool.

4. What about id numbers? For a small site, a url will appear cleanest without product ids. For a large site, an id number is needed in the URL to ensure fast page loads as each URL needs to be delivered by a database and text searches take too much time. Even if you have a small site now, what are the future growth plans? How does a tool adjust for these factors?

5. Should the brand name be included in the URL? How about the product name? How about the product description?

6. Best web page? Sites often provide numerous pages on closely related topics. Let's say you asked on Moz what is the best page to discuss web crawling tools. There are numerous articles discussing Moz's tool, Zenu, Screaming Frog and other tools. There are articles which compare web crawlers. How can a tool look at your URL and decide which page is ideal? By date? Text match? Google has billions invested in trying to provide such feedback and they clearly get it wrong at times.

There is certainly feedback a tool can offer. For example, if a url ends in .html the tool can recognize the technology extension and recommend to drop it. Very simple. With enough time and money, tools can be built to do almost anything. You presented a laundry list of needs, each of which requires a professional opinion. Even if you were to find a tool which provided responses to all of the inquiries you shared, I am highly confident the responses would not be optimal.

I hope you find this response helpful.

-Ryan

[GlobeRunner]

George,

There aren't any specific SEO migration tools out there, but there are several tools out there that will do the job. If you're an advanced SEO and know what you're doing, you can use tools like Screaming Frog's SEO spider to crawl the list of URLs and even Scrutiny (on the mac) to crawl the URLs and grab the data you need.