Why is the SEOmoz crawler crawling the old version of our website?
-
Hello,
I'm a new SEOmoz member.
On Dec. 2nd, after completely redesigning our website, we migrated to a new hosting company by switching our DNS to the new server.
The vast majority of the URLs have changed and we configured redirects of the old URLs to the new ones. Although, this task is not completed yet.
After the migration, I created an account on SEOmoz to be able to track our progress and find the issues to fix to optimize our SEO.
For some reason, in the SEOmoz reports it is the old URLs that show up. Unless the crawler does not actually crawl the pages and only uses the indexed pages to generate its report, I don't understand how could this possible.
Anyone has a clue?
When will the new URLs be indexed by SEOmoz and the major search engines?
Thanks for your help!
-
Thanks for the answers guys.
@Robert: It is indeed the initial report. Yet I did receive other reports since. Also thanks for suggestion the Screaming Frog tool. The report I get from it will be useful in my discussion with our CRM provider.
@Cyrus: I also used URI Valet. I don't understand, the 302 redirects are on picture links. I'll communicate with our CRM provider about that. Also, I don't understand how any of our old URLs could be reached crawling our website as this is a completely new website. No part was transferred over so the shouldn't be any way for the crawler to see our old URLs.
Thanks again
-
The only way that SEOmoz will crawl your old URLs is if that is what your web servers is delivering to the SEOmoz crawlers. In this case, you might want to make sure all your redirect are in place.
I did a quick check of the URL listed in your public profile using the tool URI valet, and found at least 4 redirects from the homepage, including a 302, which passes no link juice. This is definitely something you want to look into.
Generally the SEOmoz robots will crawl your site once a week, so 99% of the time you will see new infomation contained in your campaign within 7-10 day. If you suspect an error, be sure to contact the help team (help@seomoz.org)
Also, here's my favorite article on site migration. Although you didn't migrate a complete site, many of the principals still apply.
https://seogadget.co.uk/surviving-seo-site-migration/
Best of luck with your SEO!
-
Gestisoft-QC
First, SEOmoz does crawl the site it is not using indexed pages. I am assuming you set this up as a campaign and got the initial report and saw the old urls? If you are looking for links, OSE is different and does take roughly 3 to 7 weeks to show due to the amount of work involved (they are crawling the web as opposed to a site for this).
I ran your site through Screaming Frog and see the 301's, etc. You might try using Screaming Frog to see the Header Responses, etc. http://www.screamingfrog.co.uk/seo-spider/ Screaming Frog has a free download for windows or Mac and that should be sufficient to see what URLs are showing, etc. It will not give you the other info you get from moz.
I cannot see where the moz would be different, but this should at least give you a good cross reference.
If you don't get resolution there, here is the link to helpful info on moz Pro: http://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-only-crawled-one-page
You can also ask for help via help@SEOmoz.org. They are very responsive.
Hope this helps
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Moz crawler not working
Hi Moz crawler keep failing on my site with the error showing : Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. I'm not sure what am I missing out.. this is my robots.txt.. i don't think Im missing anything else.. https://www.wearefutureheads.com/robots.txt can the support team help ?
Moz Pro | | teikh0 -
Best blog practices for website
For my Insurance website blog, I use MOZ to help me find high DA authoritative sites, then either generate ideas from them, or rewrite the copy. If I rewrite the copy, I tend to pull from 2 - 3 top authoritative sites. Just so I don't get in trouble, but still offer the most concision information. _My question is, Is this ok to do? _ Secondly, I just read that on some .Gov sites the information is public, and that you can use it as long as you give credit. _My questions is, how do I tell which information is public? _ Thank you in advance 🙂
Moz Pro | | MissThumann0 -
Problem crawling a website with age verification page.
Hy every1, Need your help very urgent. I need to crawl a website that first has a page where you need to put your age for verification and after that you are redirected to the website. My problem is that SEOmoz, crawls only that first page, not the whole website. How can I crawl the whole website?, do you need me to upload a link to the website? Thank you very much Catalin
Moz Pro | | catalinmoraru0 -
What SeoMoz tool am I thinking of?
A few months ago I found a tool on Moz that did keyword link research. It is not keyword analysis either. It took a word and turned it into links such as dir:cabinet. I can't find it or remember or it. Does anyone else know? (I know this description sucks, sorry about that)
Moz Pro | | EcommerceSite0 -
How to set the crawler or reports to ignore
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
Moz Pro | | Str82u0 -
Getting your site totally indexed by SEOMOZ
Hi guys! Ijust started using SEOMOZ software and wondered how it could be that my site has over 10.000 pages but in the Pro Dashboard it only indexed about 1500 of them. I've been waiting a few weeks now but the number has been stable ever since. Is there a way to get the whole site indexed by SEOMoz software? Thanks for your answers!
Moz Pro | | ssiebn70 -
How is SeoMoz different?
In the past I heard that other SEO type crawling software like WP Gold could seriously harm your rankings in the search engines - especially Google. How is SeoMoz different ?
Moz Pro | | NeilTompkins0