Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
-
Client site is 15 + pages. New site had noindex/nofollow removed prior to last two crawls.
-
Okay I just did a test crawl with 'Screaming Frog' SEO spider - no issues
Robots.txt - not blocking it.
Headers - not blocking them.
Therefore definitely look at Keri's post as it's something SEOmoz's end, so check out that thread for some possible answers
-
Sure. The site is bayareadrainage.com . Thanks in advance.
-
If GA means Google Analytics, Google Analytics doesn't crawl pages -- it reports visits to pages that have the GA code on them. You can have the content behind a password and blocked in robots.txt and if GA code is there, it would still track people.
https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages lists some common reasons why only one page is crawled. If you still have problems with crawling, send a note to help@seomoz.org and we'll help you figure things out.
-
Are you able to post the URL? That way I, or another Mozzer could check the crawlability and just check you're not accidentally blocking robots another way?
I would also check your robots.txt to make sure you're not Disallowing all your site.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirect /label/ to /tags/
Hi guys, I have noticed loads of errors in webmaster, page not found.. /label/..... what i need to do is to a 301 redirect to /tags/... can some one tell me the redirect code to help fix this issue Regards T
Technical SEO | | Taiger0 -
Help Crawl friendliness for large site
After watching Rand's video I am trying to think of the best way to make my large site more crawl friendly. Background I have a large site with over 100k product skus and so when you get to a particular page of products there are tons of different refinements and options that help you sort the products. Most of these are noindex followed, but I was wondering if I should be nofollowing the internal links as well in order to keep bots out of those pages and going to the pages that I want them to go too. Is this a good way to handle it? Also, does anyone have good recommendations of links to posts that deal with helping the crawl friendliness of a large site? Thanks!
Technical SEO | | Gordian0 -
Site not passing page authority....
Hi, This site powertoolworld.co.uk is not passing page authority. In fact every page shows no links unless it has a link from an external source. Originally this site blocked Roger from crawling it but that block was lifted over 6 months ago. I also ran a crawl test last night and it shows the same thing. PA of 1 and no links. I would like to point out that the problem seems to be the same for all sites on the same platform. Which points me in the direction of code. for example there is a display: none tag in the ccs which is used to style where the side bar links are. It's a Blue Park platform. What could be causing the problem? Thanks in advance. EDIT Turns out that blocking the ezooms crawler stopped it from being included.
Technical SEO | | PowerToolWorld0 -
Best practice for eCommerce site migration, should I 301 redirect or match URLs on new site
Hi Guys, I have been struggling with this one for quite some time. I am no SEO expert like many of you, rather just a small business owner trying to do the right thing, so forgive me if I say something that makes no sense 🙂 I am moving our eCommerce store from one platform to another, in the process the store is getting a massive face lift. The part I am struggling with is whether I should keep my existing URL structure in place or use 301 redirects to create a cleaner looking URLs. Currently the URLs are a little long and I would love to move to a /category/product_name type format. Of course the goal is not to lose ranking in the process, I rank pretty well for several competitive phrases and do not want to create a negative impact. How would you guys handle this? Thanks, Dinesh
Technical SEO | | MyFairyTaleBooks0 -
Buying a SERPs competitor domain / site
For a specific term, we have the potential opportunity to purchase the domain (complete with site) that sits just above us in the Google search results... The domain has reasonable page authority of 49, domain authority of 38 with 168 linking root domains - 311 total links... Would the most beneficial use of the domain be to retain the site content as is and incorporate a few relevant links back to our site or... 301 the entire domain?
Technical SEO | | digitalarts1 -
Optimizing one site for multiple countries
I am working on a project, where we have one website, with a country specific domain, which is currently ranking well in local search. The client now wants to expand his business into two new countries (all english speaking) and would like to rank for the same keywords in these two new countries. The customer do not want to create new websites for the new countries. Because its a local domain and the website is setup for local search in GWT with locally hosted server, i expect challenges in optimizing for new countries without impacting the current local ranking. Question 1: What would be the recommended approach for maintaining their existing ranking on local search, while optimizing for the new countries.
Technical SEO | | petersen0 -
Getting a bunch of pages re-crawled?
I added noindex tags to a bunch (1,000+) of paginated category pages on my site. I want Google to recrawl the pages so they will de-index them. Any ideas to speed up the process?
Technical SEO | | AdamThompson0 -
How to relate two sites Domain Authority
Hi All I have been looking at advertising on some fashion blogs for our online store. Both sites have decent traffic though A is stronger than the B with more than double the traffic, Therefore given equal relevance to our business sunglasses (www.pretavoir.co.uk) it would be fair to predict that A would result in double the number of conversions.. However another interesting aspect to making a decision on which sites to advertise is their Domain Authority and how much link juice they can pass. Therefore my question is this; Putting aside any potential click through traffic, if site A Domain Authority is 70 (link to be on homepage) and site B Domain Authority is 35 is the value of site A double that of site B or is there a less linear relationship (just as with page rank). Site A are charging 500$ per year for an advertising link and Site B 100$ per year would it better business to take 5 x Site Bs or is the linkjuice passed by one DA 70 site worth more? Your thoughts would be most appreciated..
Technical SEO | | seanmccauley0