seo moz crawl diagnosis
-
Hi seomozzers,
We are creating a brand new website for a client and I would like to run an seo moz crawl to fix what has been done wrong. So my question is it ok to run an SEO moz crawl with a dev URL? Are final URLs and dev URLs will give me the same results or not?
Basically, Should I wait for getting the final URL or is it ok to run a crawl under a dev URL such as www.dev2.example.com or http://183.2564.2864?
Thank you
-
Hi Kenny,
So I have made the Bot crawl this form of url http://183.2564.2864 and detected multiple errors. So are you saying that I shouldn't rely on what the crawlers has detected? should I then assign the dev team to provide me an actual dev url before sending them any recommendations to fix errors already detected by the bot?
Thanks
-
Hey,
You could use a dev subdomain (e.g. dev.example.com) to get a crawl but we cannot crawl a specific IP address. You will alos want to make sure that our user-agent rogerbot has permission to crawl your dev site.
Hope that helps!
Kenny
-
Even under this format: http://183.2564.2864?
-
you can run it on the dev, that would be ok. cant see why not, errors to do with domain may be askew but thats all.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO - New URL structure
Hi, Currently we have the following url structure for all pages, regardless of the hierarchy: domain.co.uk/page, such as domain/blog name. Can you, please confirm the following: 1. What is the benefit of organising the pages as a hierarchy, i.e. domain/features/feature-name or domain/industries/industry-name or domain/blog/blog name etc. 2. This will create too many 301s - what is Google's tolerance of redirects? Is it worth for us changing the url structure or would you only recommend to add breadcrumbs? Many thanks Katarina
Technical SEO | | Katarina-Borovska1 -
Google only crawling a small percentage of the sitemap
Hi, The company which I work for have developed a new website for a customer, there URL is https://www.wideformatsolutions.co.uk I've created a sitemap which has 25,555 URL's. I submitted this to Google around 4 weeks ago and the most crawls that have ever occurred has been 2,379. I've checked everything I can think of, including; Speed of website Canonical Links 404 errors Setting a preferred domain Duplicate content Robots Txt .htaccess Meta Tags I did read that Matt Cutts revealed in an interview with Eric Enge that the number of pages Google crawls is roughly proportional to your pagerank. But I'm sure it should crawl more than 2000 pages. The website is based on Opencart, if anyone has experienced anything like this I would love hear from you.
Technical SEO | | chrissmithps0 -
SEO impact classifieds website
Hi, I'm part of an organization running a classifieds platform in Spain. (Mercadonline.es) We are hit by Google penalties since a few weeks, possibly caused by numerous errors we are experiencing. Most frequent errors are 404's and duplicate content (titles tags etc) since the nature of our website is dynamic. Many ads change daily, are added or removed, causing Googlebots (and others) to flag us and not being able to see our more unique content. Until what part of our platform should we be indexed? Since we have +34,000 pages indexed (mostly due to internal filter pages) I would need a systematic solution for us to display relevant and unique content, with enough usage of keywords that can bring us back up - we are actually ranked <50 on google for most of our main keywords. It is costing us precious time and money since we can only aquire our visitors (adwords etc) and not being to attract any organically. I can go in more detail with someone who can give me a bit more direction. Your answer is much appreciated! Ivor
Technical SEO | | ivordg0 -
Is this type of navigation SEO friendly?
Hi mozzers, I wanted to know if this type of navigation SEO friendly. Is it better than the regular drop down menu navigation? Thanks! Ug4MhZw.png
Technical SEO | | Ideas-Money-Art0 -
Mysterious drop in the Number of Pages Crawled
The # of crawled pages on my campaign dashboard has been 90 for months. Approximate a week ago it dropped down to 25 crawled pages, and many links went with it. I have checked with my web master, and he said no changes have been made which would cause this to happen. I am looking for suggestions on how I can go about trouble shooting this issue, and possible solutions. Thanks in advance!
Technical SEO | | GladdySEO0 -
On-Page SEO of the SEOmoz Blog Section
Hey Everyone My brain isn't working (only had 1 cup of coffee so far - #2 on it's way) this morning and I could use some help. We're creating a blog on a site for a client of ours and I've been looking at the SEOmoz blog for best practices when it comes to the implementation of pagination, canonical tags and noindex. My questions: There is no use of the canonical tag on the main blog page or any of the paginated pages but it is being used on blog post pages. Why not use it on the main blog pages as well? I'm assuming because the blog pages are always changing with different content there is not much point? Paginated pages in the category sections i.e. http://www.seomoz.org/blog/category/1?page=2 are noindexed but paginated pages in the main blog section i.e. http://www.seomoz.org/blog?page=2 are not. Is this because of a duplicate content concern since the posts in the category sections are in the main blog section as well? If that's the case, why wouldn't the main category page i.e.http://www.seomoz.org/blog/category/1 be noindexed as well? What's the reason for noindexing the "Show # Posts" pages i.e.http://www.seomoz.org/blog?show=5 ? I'm assuming another concern of duplicate content? Any insights into these questions would be greatly appreciated and would help with the implementation of our clients blog. Thanks, Ken
Technical SEO | | noBulMedia0 -
What is the most SEO friendly shopping cart for wordpress?
I'm using Thesis Theme. I know load time is important and will continue to be a bigger deal as time goes on. Any advice would be much appreciated!!
Technical SEO | | chadmorgan0 -
Magento - Google Webmaster Crawl Errors
Hi guys, Started my free trial - very impressed - just thought I'd ask a question or two while I can. I've set up the website for http://www.worldofbooks.com (large bookseller in the UK), using Magento. I'm getting a huge amount of not found crawl errors (27,808), I think this is due to URL rewrites, all the errors are in this format (non search friendly): http://www.worldofbooks.com/search_inventory.php?search_text=&category=&tag=Ure&gift_code=&dd_sort_by=price_desc&dd_records_per_page=40&dd_page_number=1 As oppose to this format: http://www.worldofbooks.com/arts-books/history-of-art-design-styles/the-art-book-by-phaidon.html (the re-written URL). This doesn't seem to really be affecting our rankings, we targeted 'cheap books' and 'bargain books' heavily - we're up to 2nd for Cheap Books and 3rd for Bargain Books. So my question is - are these large amount of Crawl errors cause for concern or is it something that will work itself out? And secondly - if it is cause for concern will it be affecting our rankings negatively in any way and what could we do to resolve this issue? Any points in the right direction much appreciated. If you need any more clarification regarding any points I've raised just let me know. Benjamin Edwards
Technical SEO | | Benj250