Web Site Migration Testing and SEO-QA Automation?
-
Hey Mozzers, Are there any good Migration-SEO-QA Tools out there?
Given a prioritized list of URLs and prioritized list of Keywords, is there a tool that can compare basic SEO factors, old URL vs. new URL, and identify all the specific gaps that need to be fixed?
Here is a basic SEO-QA acceptance checklist, for porting any website.
. . . Until the porting work is completed we cannot accept the new website.
Givens:
1. A list of the Top 100 URLs from the old site, prioritized by conversion rates, landing page traffic, and inbound links.
2. A list of the planned 404 - mapped URLs, old to new site, from the porting team.
3. A list of the current Top 200 Keywords, prioritized.
4. A good amount of SEO work has already been done, by several professionals, for the current (old) site.
**How to evaluate if the new site will be acceptable to Google? Check ON-PAGE SEO Factors... **
**. . . that is, the NEW site must be AS GOOD AS (or better than) the current (old) site,
in the eyes of Google, to preserve the On-Page SEO work already done. **Criteria:
- URLs ok? :: Is the URL mapping ok, old to new, best web page?
- LINKS ok :: Are all internal LINKS and keyword Anchor Text ported?
- TEXT ok :: On-page content, TEXT and keywords ok?
- TITLE ok :: HTML Title and title keywords ok?
- DESCRIPTION ok :: HTML Meta Description ok?
- H1, H2 ok :: HTML H1, H2 and keywords ok?
- IMG kwds :: HTML IMG and ALT keywords ok?
- URL kwds :: URL - keywords in new URLs ok?
Potential porting defects:
- Keywords in URL missing:
- Keywords in HTML Title missing:
- Keywords in Meta Description missing:
- Any internal LINKS or Link anchor text missing:
- Keywords in Page TEXT missing:
- H1, H2 missing keywords:
- HTML IMG alt-text, IMG file URLs, any missing keywords:
Notes:
- Until the porting work is completed we cannot accept the new site, or set a target date for potential cutover. There are eight (8) data items per URL, and about one hundred (100) URLs to be considered for SEO-QA before going live. We were expecting to cutover before the end of February, at the latest. There is no point in doing full QA acceptance-tests until the porting work is completed.
- QA spot-checks have found far too many defects. About 60% of the landing-page traffic comes via the top 40 URLs. With over 100 URLs to look at, it can take more than a week or two just to do SEO-QA in detail, manually, item-by-item, page-by-page, side-by-side, old vs. new.
- Spot-checks indicate a business disaster would occur unless the porting defects are fixed before going live.
_Any Migration-QA Tools?_Given a prioritized list of URLs and prioritized list of Keywords, is there a tool that can compare basic On-Page SEO factors, old URL vs. new URL, and identify most of the specific gaps that need to be fixed before going live with the new site?
_ *** Edit: Any comments on the SEO criteria, tools, or methods will be appreciated!_
-
Most cases like these are catered for on an adhoc basis, usually done by a dev team that will migrate the entire database into the new site.
There could well be a market for a porting tool that provides a checklist type system for a large scale site migration. I have seen site migrations go wrong. A lot of revenue can be lost through the plummet of organic traffic from a failed migration.
-
Hi Adam, More good points. As you wrote:
- The title, meta and content is copied across so it is blatant to Google that the redirect is in the correct place.
How to efficiently and iteratively check and re-check this work with SEO-Migration-QA Tools? ... For a list of a few hundred URLs .. old ..vs.. new? (Without a labor-intensive process.)
Surely, this is a common problem and has already been faced, and solved, by many.
If a such a Porting-QA tool does not yet exist, there must be a good market for one?
-
Hi George
It looks like most of it has been covered here but I thought I would mention a fairly simple way to do this sort of site migration. Firstly I would assume the all title tags, meta descriptions and h1 tags will be the identical to the site transitions. Understandably the URL structure may change slightly.
The number one thing is to make sure that that redirect strategy for the entire site is done correctly. This may mean crawling the entire old site using Screaming Frog to then map out exactly what URL will be redirected to what in the new site. Then your developers can structure this into redirects on the site. Depending on how many pages there are this may need tweaking to make sure there are not too many rules created to redirect the site.
In terms of making sure that each page is setup correctly from an on page point of view, this would be a very large task to accomplish. I am just typing out loud now but you could do the following:
- In Google Analytics export the top 100 (or larger depending traffic coming into your site).
- Export the rankings data that these pages are bringing traffic for organically (this will be difficult now not provided is the majority of organic keyword data but you may have historic data in GA that could help with this).
- Using the highest converting keyword for each page, use a tool such as the Moz onpage grader to fully check the page for any on page issues and an overall grade. You can then run the Moz on page checker on the new dev site and see if the grade is the same. If it is not then make the recommended changes to meet the checklist.
This is by no means an automated system but at least it is running each page using the highest converting keyword on the traffic bringing in the most traffic. This is the best way I can see this happening, it still isn't fully accurate as it is only checking the highest converting keywords but with the page potentially bringing in traffic from so many keywords it is very difficult to automate this fully.
But really the best thing to make sure is:
- The redirect strategy is bulletproof to ensure all backlinks are redirected to the new site.
- The title, meta and content is copied across so it is blatant to Google that the redirect is in the correct place.
I hope this help.
-
Hi Ryan, and thanks, again! You do raise good points on Quantitative vs. Qualitative factors.
Yes, the human component is important, but Googlebot+Algo is a mechanical device, and given the list of top 100 URLs and top 200 keywords, prioritized, it should be possible to inspect each page, old .vs. new, using an SEO-QA tool, mechanically. If the porting team invented random garbage for HTML Titles, Meta Descriptions, etc. then the new SERPs would be devastating to the business as soon as the test site goes live.
The new site may LOOK better to the naked eye than the old one, but Google will not treat the new site very kindly in the SERPs. There is always room for improvement but when a site already has had lots of SEO work done, and the porting team omits that from the new site, a disaster looms.
Checking a prioritized list of ON-Page SEO ranking factors for a prioritized list of 100 URLs and prioritized list of 200 keywords should be a fairly mechanical task. A QA tool could do it in a few minutes, while a human may take a few weeks.
**The SEO-QA-Tool could indicate a prioritized list of major SEO gaps and problems to be fixed before going live, for a HUMAN to review and take well-considered action on. **
_Update/Edit :: SEO is never really DONE, there is always room for improvement and things are always changing... even the SEO Ranking Factors. There are currently accepted authoritative opinions that have been published here, and in other places, for SEO Ranking Factors and generally accepted SEO standards. The difference between $4 per hour and $100 per hour SEO work is the quality of the SEO tools being used, and how well those SEO standards are understood, and even the experts often disagree. _
Ref: http://moz.com/search-ranking-factors and http://moz.com/blog/ranking-factors-2013 and http://moz.com/blog/future-of-search-ranking-factors and http://searchengineland.com/seotable
_Update/Edit-2: The key point here being, the SEO features of the OLD site have not been migrated to the NEW site, so the porting team has more work to do. How to QA that work without a huge labor expense? Some basic SEO-Migration-QA tools are needed, based on a sound SEO methodology such as: _http://searchenginewatch.com/article/2115729/10-Steps-to-a-Successful-SEO-Migration-Strategy
-
Hi George. These items you referred to are not mechanical as you suggest.
Title - How do you determine a "good" title? If your site is Wikipedia, a tool can certainly be used. Wikipedia titles are short and focused on a single keyword. The H1 tag and URL match or closely mirror the title. The first words of the first sentence also use the title.
For sites other than Wikipedia, how do you measure the quality of a title? Tools can share whether or not a keyword is identified in the title. They can share whether a title has more than 70 characters. If you desire optimal results for the site, other factors must be evaluated. Let's look at the title of this Q&A and just think of the first part you shared, "Web Site Migration Testing". Should it be "Web Site" or "Website"? "Testing" or "Tests". What about "Website Migration" or "Best Website Migration Tools". Or "What are the Best Website Migration Tools?" There are dozens of other variants. How will a tool decide upon the optimal title? Will it consider keyword research or existing content on the site? What about competitiveness....and so forth.
The same logic applies to the rest of the criteria you mentioned. For ALT tags the first and foremost consideration I use is "Does the ALT tag accurately represent the image? If a blind person heard the ALT tag read to them, would they be able to accurately describe the image?" As an SEO, I base image naming and content first and foremost on the user experience and then make minor adjustments to improve SEO. Theoretically, you can have a tool recognize images, provide the ALT text and adjust for SEO, but we are a long way off from that vision becoming a live tool in the field we can use.
You can certainly use tools to gather that feedback for specific elements such as title length, does a meta description exist, does an ALT tag exist, etc. As to determining whether those elements are ideally optimized to provide the best results for a particular client, that requires the human element in my experience.
-
Hi Ryan, Thanks for taking time to reply. You have some good points, but for the purely MECHANICAL factors such as HTML Title, Meta Description, IMG file names and HTML alt tags, and listing all the internal (and external) links with anchor text — side-by-side, old .vs. new — an automated On-Page SEO QA tool — would surely be very useful to assist anyone porting a website. I am surprised not to find one online as yet.
It's a kind of in-vitro A/B testing. Without such a QA procedure, going live with a test site could be disastrous, if the old site ranks well, the new site may rank very poorly.
We do have a list mapping old-url to new-url, prioritized. We have a list of keywords, prioritized.
Surely there is a niche market for some kind of automated Porting-QA reporting tool?
In a perfect world, there would be an SEO tool to predict the impact of porting before going live. Look before you leap.
-
You can use Moz, WooRank or other tools to review a site, obtain a grade along with other helpful information; however, you cannot depend on these tools to even come close to performing the type of analysis you are seeking. To use an analogy, you can use a template to draw up a will or other legal document, but if it is really important you would always work directly with a lawyer. The same applies for other fields whether it is a medical opinion or SEO advice.
Let's break down one component of your request: "URLs ok? :: Is the URL mapping ok, old to new, best web page?"
What EXACTLY is meant by URLs ok? A full breakdown would be:
1. Protocol. Are pages served as http or https? Which is best for the site?
2. Is the domain name optimal for the site?
3. What is the best URL paths for the site? If you operate a clothing e-commerce site should the primary category be the brand? gender (men vs women)? kids? something else such as clothing type (shirts / pants / suits / etc). That decision cannot be made by any tool.
4. What about id numbers? For a small site, a url will appear cleanest without product ids. For a large site, an id number is needed in the URL to ensure fast page loads as each URL needs to be delivered by a database and text searches take too much time. Even if you have a small site now, what are the future growth plans? How does a tool adjust for these factors?
5. Should the brand name be included in the URL? How about the product name? How about the product description?
6. Best web page? Sites often provide numerous pages on closely related topics. Let's say you asked on Moz what is the best page to discuss web crawling tools. There are numerous articles discussing Moz's tool, Zenu, Screaming Frog and other tools. There are articles which compare web crawlers. How can a tool look at your URL and decide which page is ideal? By date? Text match? Google has billions invested in trying to provide such feedback and they clearly get it wrong at times.
There is certainly feedback a tool can offer. For example, if a url ends in .html the tool can recognize the technology extension and recommend to drop it. Very simple. With enough time and money, tools can be built to do almost anything. You presented a laundry list of needs, each of which requires a professional opinion. Even if you were to find a tool which provided responses to all of the inquiries you shared, I am highly confident the responses would not be optimal.
I hope you find this response helpful.
-Ryan
-
George,
There aren't any specific SEO migration tools out there, but there are several tools out there that will do the job. If you're an advanced SEO and know what you're doing, you can use tools like Screaming Frog's SEO spider to crawl the list of URLs and even Scrutiny (on the mac) to crawl the URLs and grab the data you need.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why did Moz crawl our development site?
In our Moz Pro account we have one campaign set up to track our main domain. This week Moz threw up around 400 new crawl errors, 99% of which were meta noindex issues. What happened was that somehow Moz found the development/staging site and decided to crawl that. I have no idea how it was able to do this - the robots.txt is set to disallow all and there is password protection on the site. It looks like Moz ignored the robots.txt, but I still don't have any idea how it was able to do a crawl - it should have received a 401 Forbidden and not gone any further. How do I a) clean this up without going through and manually ignoring each issue, and b) stop this from happening again? Thanks!
Moz Pro | | MultiTimeMachine0 -
Question about SEO and domains
Hello, Over time we have purchased many domain names that relate to our business. Let us say we sell chairs of type A, chairs of type B, Tables of Type C etc. We also have a main web site (main domain) which has been functioning for many years and is ranked well but not great. Since we have domain chairtypeA.com and chairtypeB.com I was thinking of putting them to good use as follows. STRATEGY A a) Create the domain as a parallel domain but is mainly informational - it talks about chair type A, its history, what to look for etc. It has the same IP address as main domain b) Have all the links on the page be directly linked to the MAIN site- so if you are on page for ChairtypeA and want to see it in a set you just click a link and it takes you to the main site. c) The look and feel of the ChairtypeA site is exactly the same as the main site - in fact except for the Heading, title and content the whole framework is from the main site. My hope is that if people are looking for chairtypeA, we would rank and attract customers. Plus it would use up some of the domain names we have purchased. THE OTHER ALTERNATIVE (Strategy B) is a) Create a domain for chairtypeA parallel to main domain (same IP address) b) redirect it to subfolder in main domain c) refer to the pages also as maindomain/chairtypeA (here is where it is no longer purely parallel) I have 3 objectives a) Use my purchased domain names b) Focus on certain products (liek ChairtypeA) c) Overall increase visibility and ranking Please give me any feedback which indicates whether Strategy A or Strategy B is more effective, or if any has negative consequences. Or you could suggest Strategy C - a completely new approach. Thanks Sam
Moz Pro | | samgold0 -
Redirects, SEO and More
Website errors remittent of Chernobyl. First of all, I ask a lot of questions of the community and I don't do a lot of answering. For that, I am sorry! I own a small SEO company and the majority of my clients are upstarts just like myself. We try to do the best we can on a very limited budget.
Moz Pro | | underdogmike
With that said, the client website in question, http://bestdefensega.com has been a struggle to rank in the metro atlanta lawyer market.
Here's the story. My client is a new solo attorney that practices criminal law and personal injury law. We launched bestdensega.com to focus on criminal defense and my client has also paid AVVO for a website as well. We recently decided to make a monster website that would have 50+ pages and be representative of all that he practices.
I recently put a temporary redirect on bestdefensega.com and pointed it to the avvo website while I was rebuilding the website. I did this using a plugin.
This morning I decided that I wanted to remove the redirect, so I deactivated and deleted the plugin. The redirect is still in effect. I suspect that this may be a cacheing scenario.
How do I undo a redirect?
Also, I noticed that the AVVO site has all of a sudden shot up to the top of the search results and bestdefnesega.com has dropped past the 3rd page for our keywords. Have I lost all of bestdefensega.com's link juice? Also, bestdefensega.com has a spam score of 7. Why? There are no shady links and everything I do is on the up and up. Also Open Site Explorer says that bestdefensega.com doesn't have any inbound links.
Google WebMaster Tools says that the site has of 500 inbound links.. Frustrated, Thanks0 -
Where do I post this list of hacked sites?
Hey guys, Fairly new to SEOmoz but loving it so far. I was working on a new clients site a noticed some spammy links added right before the tag. Used Open site explorer to list the domains linking to the url and found nearly 300 unsuspecting domains. Some like heartresearch.com.au which just drives me craaazy, I have already emailed them. Below is the list. http://www.opensiteexplorer.org/links.html?group=0&page=3&site=www.rhcie.com Short of emailing every single person can anyone suggest a forum or such that would be helpful for posting this information ? I know it's just a few links but it is frustrating to me and If I can do something about it I would like to. Thanks in advance. Jason
Moz Pro | | RedshiftWebDesign0 -
Open Site Explorer
Hi - open site explorer always showed backlinks to my site's blog for Wordpress, a couple of edu sites and others that ranked somewhat high. These are no longer being shown. However, they do show up on some other backlink check tools. Does anyone know why and if this should be a concern? Thanks. Don www.uniquegiftsanddecor.com
Moz Pro | | uniquegifts-2778790 -
OPen Site Explorer listing criteria?
Checking link data for a potential client who has given me access to their websmaster tools account, I've noticed that Google list over 150,000 "Links to Your Site" but yet OSE only reports a tiny fraction of this number.
Moz Pro | | G-DC
I have manually verified that there are in fact direct links from the sites that are listed in Webmaster Tools but not in OSE. Does OSE discard some links? Is there a reporting lag? Does anyone know why there would be such a large discrepancy?0 -
Cant understand Seo moz analysis
Did a semoz analysis of site www.1stclassdriving.co.uk and it came up with 484 pages with duplicate titles and 484 with duplicate content. Also gave me a 531st ranking for search term "Driving Lessons" (with many smaller schools with better rankings)- so I think that I may have a problem 1-Duplicate titles - I have many area pages - one for each of the post codes that we cover (some 40 odd) Each page is titled "Driving Lessons in "xxx" where xxx is the area.
Moz Pro | | Brian_Worger
Plus each Instructor has their own page which is tiled "Driving Lessons in xxx" where xxx is the more local towns/areas covered.
Is this the use of the term Driving Lessons xxx on each page the problem? The report doesnt seem to make this clear -it simply lists each page on the site 2- Duplicate content puzzles me as the content does differ although each instructor page has probably 80% content derived from a master page. Attach screenshot - hope someone can help Many thanx Brian0 -
Backlinks using Open Site Explorer
Hey guys, So I was doing some backlink analysis for a client and was utilizing Open Site Explorer, which stated that I had roughly 2,111 External Followed Links. However, I can't seem to find a way to export all 2,111 backlinks. I wanted to make sure if the links I had acquired had been indexed and crawled, and picked up, but I can't seem to access the full list on a domain level, only on the subdomain level. Anybody have any suggestions or advice? Any would be helpful! Best, James
Moz Pro | | sixspokemedia0