SEOMOZ Crawler unicode bug
-
for the last couple of weeks the SEOMOZ crawls my homepage only and gets 4xx error for most of the URL's.
the crawler have no issues with English url's only with the unicode(Hebrew) ones.
this is what is see in the csv export for the crawl (one sample) :
http://www.funstuff.co.il/׳ž׳¡׳
׳‘׳×-׳¨׳•׳•׳§׳•׳× 404 text/html; charset=utf-8you can see that the URL is Gibberish
please help.
-
Hey Asaf,
Thanks for writing in.
We have a known issue where Hebrew isn't parsed right by our crawler so it has caused issues in the past. The issues have been intermittent but they can affect the data you see. Sorry about that. Our engineers have been working to get a fix out there for the Hebrew character set, so stay tuned.
Best,
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Functionality of SEOmoz crawl page reports
I am trying to find a way to ask SEOmoz staff to answer this question because I think it is a functionality question so I checked SEOmoz pro resources. I also have had no responses in the Forum too it either. So here it is again. Thanks much for your consideration! Is it possible to configure the SEOMoz Rogerbot error-finding bot (that make the crawl diagnostic reports) to obey the instructions in the individual page headers and http://client.com/robots.txt file? For example, there is a page at http://truthbook.com/quotes/index.cfm month=5&day=14&year=2007 that has – in the header -
Moz Pro | | jimmyzig
<meta name="robots" content="noindex"> </meta name="robots" content="noindex"> This page is themed Quote of the Day page and is duplicated twice intentionally at http://truthbook.com/quotes/index.cfm?month=5&day=14&year=2004 and also at http://truthbook.com/quotes/index.cfm?month=5&day=14&year=2010 but they all have <meta name="robots" content="noindex"> in them. So Google should not see them as duplicates right. Google does not in Webmaster Tools.</meta name="robots" content="noindex"> So it should not be counted 3 times? But it seems to be? How do we gen a report of the actual pages shown in the report as dups so we can check? We do not believe Google sees it as a duplicate page but Roger appears too. Similarly, one can use http://truthbook.com/contemplative_prayer/ , here also the http://truthbook.com/robots.txt tells Google to stay clear. Yet we are showing thousands of dup. page content errors when Google Webmaster tools as shown only a few hundred configured as described. Anyone? Jim0 -
CSV reports in SEOmoz
Hello, I would like to export the reports from SEOmoz to an Excel sheet. However when I downoad the report and open it, all the information is random and is hard to work on it. Since Im not an excel expert, I have to ask if there is an Excel sheet ready to receive the SEOmoz reports. Tks for the help, Regards, PP
Moz Pro | | PedroM0 -
Ranking Differs between SEOmoz and Google Webmaster?
My Google webmaster ranking for keywords differs significantly from SEOmoz. For example, Webmaster say 8 and SEOmoz says 1...thoughts?
Moz Pro | | mjcarrjr0 -
404 errors in SEOMoz crawl tool
I currently have several 404 errors in the latest crawls from SEOMoz. Here is an example of the error. http://dealerplatform.com/blog/2011/10/23/videos-for-auto-dealers/www.dealerplatform.com/ In all cases the error is a result of www.dealerplatform.com being added to the real url. Anyone seen this before? The site is a wordpress mutlisite. I don't see where this incorrect link is showing up anywhere on the website. Any advice would be helpful. Thanks
Moz Pro | | Chris_Gregory0 -
How do I find the corresponding duplicate content pages from my SEOmoz report?
Once I have run my report and the duplicate content pages come up, is there a way to find out which pages have the duplicate content on them? I have one URL but where can I find the duplicate content that corresponds to it? Thanks Barry
Moz Pro | | MrBarrytg0 -
Is there a way to manually update the keyword rankings tool in SEOmoz?
It appears that our keyword rankings update weekly on Thursdays. Is this the same across the board or is there a way to manually run an update?
Moz Pro | | Motivators0 -
Are the Ranking Changes on SEOmoz ranking reports weekly or life of the keyword?
I am referring to the area where it says ranking changes and where it has a green and a red arrow for improved and declined.
Moz Pro | | JGar-2203710 -
How do i get to know th pages crawled by SEOMOZ?
My SEOMOZ campaign says that "n" number of pages were crawled. How do i get access to the list of the pages crawled by SEOMOZ?
Moz Pro | | IM_Learner0