Ajax #! URL support?

[oneactlife]

Hi Moz,

My site is currently following the convention outlined here:

https://support.google.com/webmasters/answer/174992?hl=en

Basically since pages are generated via Ajax we are setup to direct bots that replace the #! in a url with ?escaped_fragment to cached versions of the ajax generated content.

For example, if the bot sees this url:

http://www.discoverymap.com/#!/California/Map-of-Carmel/73

it will replace it will instead access the page:

http://www.discoverymap.com/?escaped_fragment=/California/Map-of-Carmel/73

In which case my server serves the cached html instead of the live page. This is all per Googles direction and is indexing fine.

However the MOZ bot does not do this. It seems like a fairly straight-forward feature to support. Rather than ignoring the hash, you look to see if it is a #! and then try to spider the url replaced with ?escaped_fragment. Our server does the rest.

If this is something MOZ plans on supporting in the future I would love to know. If there is other information that would be great.

Also, pushstate is not practical for everyone due to limited browser support, etc.

Thanks,

Dustin

Updates:

I am editing my question because it won't let me respond to my own question. It says I need to sign up for MOZ analytics. I was signed up for Moz Analytics?! Now I am not? I responded to my invitation weeks ago?

Anyway, you are misunderstanding how this process works. There is no site-map involved. The bot reads this URL on the page:

http://www.discoverymap.com/#!/California/Map-of-Carmel/73

And when it is ready to spider the page for content it, it spider's this URL instead:

http://www.discoverymap.com/?escaped_fragment=/California/Map-of-Carmel/73

The server does the rest, it is simply telling Roger to recognize the #! format and replace it with

?escaped_fragment

Though I obviously do not know how Roger is coded but it is a simple string replacement.

Thanks.

[Abe_Schmidt]

Hello Dustin, this is Abe on the Moz Help team.

This question is a bit intricate, I apologize if i am not reading your question correctly.

With AJAX content like this, I know Google's full specifications

https://developers.google.com/webmasters/ajax-crawling/docs/specification

indicate that the #! and ?escaped_fragment= technique works for their crawlers. However, Roger is a bit picky and isn't robust enough yet to use only the sitemap as the reference in this case. Luckily, one of our wonderful users came up with a solution using pushState() method. Click here:

http://www.moz.com/blog/create-crawlable-link-friendly-ajax-websites-using-pushstate

to find out how to create crawl-able content using pushState . This should help our crawler read AJAX content. Let me know if this information works for you!

I hope this helps