Why is the SEOmoz crawler crawling the old version of our website?
-
Hello,
I'm a new SEOmoz member.
On Dec. 2nd, after completely redesigning our website, we migrated to a new hosting company by switching our DNS to the new server.
The vast majority of the URLs have changed and we configured redirects of the old URLs to the new ones. Although, this task is not completed yet.
After the migration, I created an account on SEOmoz to be able to track our progress and find the issues to fix to optimize our SEO.
For some reason, in the SEOmoz reports it is the old URLs that show up. Unless the crawler does not actually crawl the pages and only uses the indexed pages to generate its report, I don't understand how could this possible.
Anyone has a clue?
When will the new URLs be indexed by SEOmoz and the major search engines?
Thanks for your help!
-
Thanks for the answers guys.
@Robert: It is indeed the initial report. Yet I did receive other reports since. Also thanks for suggestion the Screaming Frog tool. The report I get from it will be useful in my discussion with our CRM provider.
@Cyrus: I also used URI Valet. I don't understand, the 302 redirects are on picture links. I'll communicate with our CRM provider about that. Also, I don't understand how any of our old URLs could be reached crawling our website as this is a completely new website. No part was transferred over so the shouldn't be any way for the crawler to see our old URLs.
Thanks again
-
The only way that SEOmoz will crawl your old URLs is if that is what your web servers is delivering to the SEOmoz crawlers. In this case, you might want to make sure all your redirect are in place.
I did a quick check of the URL listed in your public profile using the tool URI valet, and found at least 4 redirects from the homepage, including a 302, which passes no link juice. This is definitely something you want to look into.
Generally the SEOmoz robots will crawl your site once a week, so 99% of the time you will see new infomation contained in your campaign within 7-10 day. If you suspect an error, be sure to contact the help team (help@seomoz.org)
Also, here's my favorite article on site migration. Although you didn't migrate a complete site, many of the principals still apply.
https://seogadget.co.uk/surviving-seo-site-migration/
Best of luck with your SEO!
-
Gestisoft-QC
First, SEOmoz does crawl the site it is not using indexed pages. I am assuming you set this up as a campaign and got the initial report and saw the old urls? If you are looking for links, OSE is different and does take roughly 3 to 7 weeks to show due to the amount of work involved (they are crawling the web as opposed to a site for this).
I ran your site through Screaming Frog and see the 301's, etc. You might try using Screaming Frog to see the Header Responses, etc. http://www.screamingfrog.co.uk/seo-spider/ Screaming Frog has a free download for windows or Mac and that should be sufficient to see what URLs are showing, etc. It will not give you the other info you get from moz.
I cannot see where the moz would be different, but this should at least give you a good cross reference.
If you don't get resolution there, here is the link to helpful info on moz Pro: http://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-only-crawled-one-page
You can also ask for help via help@SEOmoz.org. They are very responsive.
Hope this helps
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site Crawl 4xx Errors?
Hello! When I check our website's critical crawler issues with Moz Site Crawler, I'm seeing over 1000 pages with a 4xx error. All of the pages that are showing to have a 4xx error appear to be the brand and product pages we have on our website, but with /URL at the end of each permalink. For example, we have a page on our site for a brand called Davinci. The URL is https://kannakart.com/davinci/. In the site crawler, I'm seeing the 4xx for this URL: https://kannakart.com/davinci/URL. Could this be a plugin on our site that is generating these URLs? If they're going to be an issue, I'd like to remove them. However, I'm not sure exactly where to begin. Thanks in advance for the help, -Andrew
Moz Pro | | mostcg0 -
No Crawl data in dashboard
For the second straight week, I have had no crawl data in my dashboard. It seems like the crawler erased all my results in the pro dashboard. Is there a way to manually recrawl my site, since I will have to wait another week to see if it comes back to earth? Thanks
Moz Pro | | bedwards0 -
Old data in OSE?
Hello, I know the Mozscape was recently updated, but it's been a week and my site data ( DA, number of backlinks) in Open Site Explorer is still the same. The seomoz toolbar and my pro tools show my DA has increased ( I don't know of another way to verify my backlink count, so I'm not sure on that front). Any idea of the reason behind the disparity? Thanks.
Moz Pro | | richje0 -
Why does SEOMoz crawler ignore robots.txt?
The SEOMoz crawler ignores robots.txt It also "indexes" pages marked as noindex. That means it is filling up the reports with things that don't matter. Is there any way to stop it doing that?
Moz Pro | | loopyal0 -
Seomoz Toolbar - Mozilla 6.0
Hi there, when will the toolbar be compatible with Mozilla 6.0? Thank you for your feedback. BR, Ive
Moz Pro | | NeroSolutions0 -
Errors on my Crawl Diagnostics
I have 51 errors on my Crawl Diagnostics tool.46 are 4xx Client Error.Those 4xx errors are links to products (or categories) that we are not selling them any more so there are inactive on the website but Google still have the links. How can I tell Google not to index them?. Can those errors (and warnings) could be harming my rankings (they went down from position 1 to 4 for the most important keywords) thanks,
Moz Pro | | cardif0 -
How is SeoMoz different?
In the past I heard that other SEO type crawling software like WP Gold could seriously harm your rankings in the search engines - especially Google. How is SeoMoz different ?
Moz Pro | | NeilTompkins0 -
How to track with SEOMOZ a website in several language
Hi, We have a customer with a website in EN, FR and ES. They used Joomfish, so each language is in a subdirectory : sitename/en sitename/fr sitename/es and they want their website to be well placed on the web for all that languages and countries: English, French, Spanish, German and Italian. It is a website for specific affiliation, that's why there is no barriers. What I need to do to use the best way SEOmoz. For the moment I created one campaign following Google US, google Germany and Google France. To go deeper, I would need to create different campaigns in my account? And also, your robot will be able to recognize the different subdirectories and languages? And to improve the SEO of this website, it wouldn't be better to have 3 domains name, one for each country? Thanks a lot in advance for your answer, Anne
Moz Pro | | ahernoux1