Website not crawled
-
i added website www.nsale.in in add campaign, it shows only 1 page crawled. but its working fine for other sites, any idea why it failed ?
-
This page references several reasons why only one page may be crawled. If it isn't any of these reasons, please contact our help desk. Thanks!
-
just looked at the other site you say is working... it has a much simpler robots file and no delay. just sayin' //
-
That's one of the bigger robots files I've seen in awhile..
You sure that crawl delay isn't messing things up? Not sure but that'd be my first guess.
-
its from day one. i have other domains on the same server www.litetab.in, its crawling properly. not sure why www.nsale.in alone doesnt work.
-
How long has it been up? Also, did you add the site from its root?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I can't crawl the archive of this website with Screaming Frog
Hi I'm trying to crawl this website (http://zeri.info/) with Screaming Frog but because of some technical issue with their site (i can't find what is causing it) i'm able to crawl only the first page of each category (ex. http://zeri.info/sport/) and then it will go to crawl each page of their archive (hundreds of thousands of pages) but it won't crawl the links inside these pages. Thanks a lot!
Technical SEO | | gjergjshala0 -
E-commerce website
Hi, I have to do a SEO optimization on a huge e-commerce website, but i don´t know if i have to focus in Schema or meta tags ( page title, meta description) which of them is more important? how can I optimize the website? Thanks
Technical SEO | | AbacoDigital1 -
Redesiging website and how to prevent a SEO disaster?
Hey guys, I'm in the process of re designing my company website. This website runs in Kentico CMS and now I'm moving to drupal CMS. The main reason for the move is that I need a more flexible design with responsiveness and integrate better social aspects. Currently I'm having to deal with a developer whom can't be bothered to update the site. I also find kentico extremely difficult to deal with. The site has good Google rankings and I was thinking about setting up some 301 redirects when the move is completed. Now the question is this site is been live for over 2 years and I have tons of blog posts sitting in here. How can I get these moved? When this is done will they lose their 'age' ? What if I re create the blog posts as it is and then setup 301 re-directs on this? I have G+ authorship and will they be affected by this? Last question is having a responsive site will affect my SEO rankings once moved? I heard the 301 is not considered as a 'safe' thing with Google so what are my options? If anyone can share a link(s) to site upgrades/moving to new design tutorials/best practices articles based on SEO ill be grateful. Thank you very much.
Technical SEO | | Suganthan0 -
Blocking https from being crawled
I have an ecommerce site where https is being crawled for some pages. Wondering if the below solution will fix the issue www.example.com will be my domain In the nav there is a login page www.example.com/login which is redirecting to the https://www.example.com/login If I just disallowed /login in the robots file wouldn't it not follow the redirect and index that stuff? The redirect part is what I am questioning.
Technical SEO | | Sean_Dawes0 -
Problem wth Crawling
Hello, I have a website http://digitaldiscovery.eu here in SEOmoz. Its strange since the last week SEOmoz is crawling only one page! And before it was crwaling all the pages. Whats happening? Help SEOmoz! :))
Technical SEO | | PedroM0 -
Absolute of Relative Internal Website Links
Hi, I am not sure what is considered best practice when linking between pages on the same site - absolute or relative: Link Or Link I notice a lot of CMS systems (WordPress) use the absolute method - is there a reason? Any help much appreciated. Barney.
Technical SEO | | barnst0 -
Hosting in the US for an Australian website
Hi, I was wondering whether it's still important for SEO to host a website in the same country as you're targetting. We host a fair few Australian sites and we're looking for a new webhost at the moment. Should we only consider hosting on Australian servers or is it worth looking at servers elsewhere? One of the main reasons we're thinking about US hosting is that we can get a machine that is 3x more powerful than we can get in Australia for the price. This performance increase should go some way to minimizing the effect of the latency increase. Thanks, Nick
Technical SEO | | MarketingResults0 -
Crawl Errors
Okay, I was just in my Google Webmaster Tools and was looking at some of the stats. I have 1354 "not found" pages google says. Many of these URL's are bizarre. I don't know what they are. Others I do know. What should I do about this? Especially all the URL's I don't even know what they are?
Technical SEO | | azguy0