Website not crawled
-
i added website www.nsale.in in add campaign, it shows only 1 page crawled. but its working fine for other sites, any idea why it failed ?
-
This page references several reasons why only one page may be crawled. If it isn't any of these reasons, please contact our help desk. Thanks!
-
just looked at the other site you say is working... it has a much simpler robots file and no delay. just sayin' //
-
That's one of the bigger robots files I've seen in awhile..
You sure that crawl delay isn't messing things up? Not sure but that'd be my first guess.
-
its from day one. i have other domains on the same server www.litetab.in, its crawling properly. not sure why www.nsale.in alone doesnt work.
-
How long has it been up? Also, did you add the site from its root?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
If few backlinks got deleted how they affect website?
Dear Experts,
Technical SEO | | jfdagborrbg
This is Julia and I am SEO Executive at J com. I would like to ask that If we are building 150+ quality backlinks per month for any website and 30 backlinks are deleting each month from 150 backlinks so It would be OK (natural) OR it would affect website ranking. I am looking forward to your response0 -
Multilingual website
My website is https://www.india-visa-gov.in and we are doing multilingual. There are three options 1. TLD eg india-visa-gov.fr (French) india-visa-gov.de (German) 2. Subdomain eg: fr.india-visa-gov.in (French) de.india-visa-gov.in (German) 3. Folders https://www.india-visa-gov.in/fr/ (French) https://www.india-visa-gov.in/de/ (German) We have tried the 3rd option but need to know whether its better or not for the long term health from SEO. Does the MOZ DA carry better in Subdomain or TLD or Folders? What does MOZ recommend to maintain DA? Thanks
Technical SEO | | amitdipsite150220200 -
Google Crawling Issues! How Can I Get Google to Crawl My Website Regularly?
Hi Everyone! My website is not being crawled regularly by Google - there are weeks when it's regular but for the past month or so it does not get crawled for seven to eight days. There are some specific pages, that I want to get ranked but they of late are not being crawled AT ALL unless I use the 'Fetch As Google' tool! That's not normal, right? I have checked and re-checked the on-page metrics for these pages (and the website as a whole, backlinking is a regular and ongoing process as well! Sitemap is in place too! Resubmitted it once too! This issue is detrimental to website traffic and rankings! Would really appreciate insights from you guys! Thanks a lot!
Technical SEO | | farhanm1 -
Does Google see the connection between our 2 websites?
We have a main website and we created a satellite site to support the original one with backlinks. Can I add both sites to the same Google Analytics profile? My programmer said that there is no reason to use a new GA profile for the satellite site, since Google will see the connection between the 2 websites via scripts, Google Plus buttons and other programmed solutions. So, is there a reason to use a new GA account for the satellite site (and later the new satellite sites) as well?
Technical SEO | | Romaine0 -
Website redirects
We consolidated websites. All the international sites have been brought under the roof of our mothership site based in the US: www.crisisprevention.com ... We mapped out all of the URLs and where they should be redirected. However, if someone types in, say, www.crisisprevention.co.uk it redirects to the mothership site, BUT the old URL hangs around no matter what page you navigate to. I feel like it has duplicate content ramifications or worse. I would like opinions on this, so I can take my findings to IT and figure out a solution. Here’s another example: http://www.positive-options.co.uk and another http://www.positive-options.com
Technical SEO | | spackle0 -
Portfolio website: reciprocal backlinks and redirects
My company specializes in website design and development, and as such, the portfolio section on our website has lots of links to websites we've built. Generally, the way that we do this is that our portfolio sections links to URLs with a naming scheme as follows: http://ourcompany.com/client1 which is either a redirect to http://client1.com if the site is still up, or a version hosted on our website (http://ourcompany.com/client1) if the client is no longer hosting their site. In addition, most of our client sites contain a reciprocal link back to ourcompany.com, usually with the text "Website design and development by OurCompany". I have two questions about this arrangement: Might the reciprocal backlinks from the client sites be affecting the SEO results for ourcompany.com? I know that reciprocal backlinks can be occasionally problematic. If I do a search for "site:ourcompany.com", I'll get results for all of the client sites that we're hosting at http://ourcompany.com/client1, because I haven't excluded them in robots.txt. Of course, the content of these sites has little to do with that of our company, because we have a diverse array of clients and none of them do web design and development (which is the content focus of our site). Might this seemingly unrelated content be affecting our results? Should the client sites be excluded in robots.txt or via nofollow? Thanks very much!
Technical SEO | | fdepino0 -
Resurrecting a heavily duplicated Joomla website.
The site in question is stanwix.info. I set this up years ago as a community project before I had even heard of seo. It carries quite a bit of content but has appallingly "SEO Unfriendly" URLs some content is recorded as duplicate 3 or more times because of these dynamic URLs according to the crawl statistics returned by SEO Moz. Currently it ranks well for the keyword stanwix - not too surprising as the only significant local competition is a large commercial holiday complex located several miles away out on the coast. As a sort of showcase, I have been trying to come up with a strategy to re-optimise stanwix.info so that it can outdo the commercial site stanwix.com I am pretty sure that step 1 involcves upgrading from the current archaic version of Joomla to the latest release even though that will break some of the modules currently installed. My pal who actually coded the site did a fair bit of hacking of the modules to get them to work properly. OR do I simply bin it and start a fresh site consigning years worth of community content to the darkness and replace with fresh material - I'm not keen on this option as I have put a lot of heart and soul into this project. I am in a quandary here as the problems are so numerous that whatever I do will break the site for some time. The reason I want to keep the site is because the business directory provides a useful locally relevant source of links for other clients and I am also going to set up a separate website for the community centre on a subdomain - this time running in wordpress which I do have a fair bit of experience with. Your advice for my addled brain would be much appreciated.
Technical SEO | | SkiBum0 -
Duplicate Content within Website - problem?
Hello everyone, I am currently working on a big site which sells thousands of widgets. However each widget has ten sub widgets (1,2,3... say) My strategy with this site is to target the long tail search so I'm creating static pages for each possibly variation. So I'll have a main product page on widgets in general, and also a page on widget1, page on widget2 etc etc. I'm anticipating that because there's so much competition for searches relating to widgets in general, I'll get most of my traffic from people being more specific and searching for widget1 or widget 7 etc. Now here's the problem - I am getting a lot of content written for this website - a few hundred words for each widget. However I can't go to the extreme of writing unique content for each sub widget - that would mean 10's of 1,000's of articles. So... what do I do with the content. Put it on the main widget page was the plan but what do I do about the sub pages. I could put it there and it would make perfect sense to a reader and be relevant to people specifically looking for widget1, say, but could there be a issue with it being viewed as duplicate content. One idea was to just put a snippet (first 100 words) on each sub page with a link back to the main widget page where the full copy would be. Not sure whether I've made myself clear at all but hopefully I have - or I can clarify. Thanks so much in advance David
Technical SEO | | OzDave0