Ajax4SEO and rogerbot crawling
-
Has anyone had any experience with seo4ajax.com and moz?
The idea is that it points a bot to a html version of an ajax page (sounds good) without the need for ugly urls. However, I don't know how this will work with rogerbot and whether moz can crawl this. There's a section to add in specific user agents and I've added "rogerbot".
Does anyone know if this will work or not? Otherwise, it's going to create some complications. I can't currently check as the site is in development and the dev version is noindexed currently.
Thanks!
-
Hi Philip!
This question is a bit intricate.
With AJAX content like this, I know Google's full specifications
https://developers.google.com/webmasters/ajax-crawling/docs/specification
indicate that the #! and ?escaped_fragment= technique works for their crawlers. However, Roger is a bit picky and isn't robust enough yet to use only the sitemap as the reference in this case. Luckily, one of our wonderful users came up with a solution using pushState() method. Click here:
http://www.moz.com/blog/create-crawlable-link-friendly-ajax-websites-using-pushstate
to find out how to create crawl-able content using pushState. The only other thing I can think of is to run a crawl test if the site is live. You'll have to remove the noindex tag, but updating the robots.txt to allow rogerbot but have a wildcard disallow for the other crawlers should still keep the site from being indexed.
Hopefully this will help!
Best,
Sam
Moz Helpster
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Rookie question re Moz Crawl errors after deleting a property from console.
Hi all, I stupidly removed the "http" url of my one website a few days back (it is one of three, the other two being the https), then re-added it around a day later and, while google console isn't reporting back any errors, Moz Crawl is going to town on this now for one critical "4xx" issues, canonicals and various other content issues that I addressed days previously...last Moz crawl performed an hour ago, url deleted and re-added two days ago. I have resubmitted a sitemap, will this smooth itself out or shall I go and make changes? Many thanks in advance.
Moz Pro | | UkPod0 -
How do a run a MOZ crawl of my site before waiting for the scheduled weekly crawl?
Greetings: I have just updated my site and would like to run a crawl immediately. How can I do so before waiting for the next MOZ crawl? Thanks,
Moz Pro | | Kingalan1
Alan0 -
How to find page with the link that returns a 404 error indicated in my crawl diagnostics?
Hi Newbie here - I am trying to understand what to do, step by step, after getting my initial reports back from seomoz. The first is regarding the 404 errors shown as high priority to fix, in crawl diagnostics. I reviewed the support info help on the crawl diagnostics page referring to 404 errors, but still did not understand exactly what I am supposed to do...same with the Q&A section when I searched how to fix 404 errors. I just could not understand exactly what anyone was talking about in relation to my 404 issues. It seems I would want to find the page that had the bad link that sent a visitor to a page not found, and then correct the problem by removing the link, or correcting and re-uploading the page being linked to. I saw some suggestions that seemed to indicate that seomoz itself will not let me find the page where the bad link is and that I would need to use some external program to do this. I would think that if seomoz found the bad page, it would also tell me what page the link(s) to the bad page exists on. A number of suggestions were to use a 301 redirect somehow as the solution, but was not clear when to do this versus, just removing the bad link, or repairing the page the link was pointing to. I think therefore my question is how do I find the links that lead to 404 page not founds, and fix the problem. Thanks Galen
Moz Pro | | Tetruss0 -
I have a Rel Canonical "notice" in my Crawl Diagnostics report. I'm presuming that means that the spider has detected a rel canonical tag and it is working as opposed to warning about an issue, is this correct?
I know this seems like a really dumb question but the site I'm working on is a BigCommerce one and I've been concerned about canonicalisation issues prior to receiving this report (I'm a SEOmoz pro newbie also!) and I just want to be clear I am reading this notice correctly. I presume this means that the site crawl has detected the rel canonical tag on these pages and it is working correctly. Is this correct?? Any input is much appreciated. Thanks
Moz Pro | | seanpearse0 -
Set crawl frequency
Current crawl frequency is weekly, is it possible for me to set this frequency our-self?
Moz Pro | | bhanu22170 -
How long is a full crawl?
It's been now over 3 days that the dashboard for one of our campaigns shows "Next Crawl in Progress!". I am not complaining about the length... but I have to agree that SEOMoz is quite addictive, and it's quite frustrating to see that everyday 🙂 Thanks
Moz Pro | | jgenesto0 -
Why did SEOMoz only crawl 1 page?
I have multiple campaigns and on a few of them SEOMoz has only crawled one page. I think this may have to do with how I set up the campaign. How do I get SEOMoz to crawl more than one page on these campaigns.
Moz Pro | | HermanAdvertising0 -
Crawl complete, but nothing changed?
Hi everyone, According to my account, the crawl diagnostics were completed yesterday. However, the duplicate page titles that it mentions aren't correct. The changes that I implemented several days ago are not being shown in the report. When I click the duplicate page title links, the latest date in the graph is 3/26. However, it says the crawl was completed on 3/30. Does it take a few days for the reports to match what the crawl actually discovered?
Moz Pro | | beeneeb0