Website not crawled
-
i added website www.nsale.in in add campaign, it shows only 1 page crawled. but its working fine for other sites, any idea why it failed ?
-
This page references several reasons why only one page may be crawled. If it isn't any of these reasons, please contact our help desk. Thanks!
-
just looked at the other site you say is working... it has a much simpler robots file and no delay. just sayin' //
-
That's one of the bigger robots files I've seen in awhile..
You sure that crawl delay isn't messing things up? Not sure but that'd be my first guess.
-
its from day one. i have other domains on the same server www.litetab.in, its crawling properly. not sure why www.nsale.in alone doesnt work.
-
How long has it been up? Also, did you add the site from its root?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ecommerce Preferred URL Structure for Printing Website
Hello Mozers! We are adding an ecommerce functionality to our existing website.
Technical SEO | | CheapyPP
Our company offers a wide range of commercial printing and mail services. We have done a pretty good job over the years in building content both in terms of our print offerings and blog section highlighting those offerings. We have finally bit the bullet and have decided to add end-to end ecommerce functionality. Users will be able to price, pay, upload and order thru our website. My question to the community becomes which sub folder do we use?
The ecommerce functionality is a third part software and needs to sit in a sub folder and we can't seem to find a good fit. Most of our content pages for print items are something like this www.website/printing/ - pillar page examples of url structure for sub pages www.website/printing/flyer-printing/
www.website/printing/booklet-printing/
www.website/printing/door-hangers/
www.website/printing/business-cards/ Options would be order-printing/ or prints/ So we we thinking /orders/ would be the best but not certain and wanted some feedback from the community. If we did go this route the url structure would be: order/business-cards this would be the default econ page order/business-cards/full-uv-coaing-both-sides individual product page What are your thoughts? CH0 -
Website title duplicates in SEO description
Hi - My website title (company name) repeats in the SEO description. My host service is Square Space. How do I fix this?
Technical SEO | | Peeker
Thanks! Paula board-directors0 -
I can't crawl the archive of this website with Screaming Frog
Hi I'm trying to crawl this website (http://zeri.info/) with Screaming Frog but because of some technical issue with their site (i can't find what is causing it) i'm able to crawl only the first page of each category (ex. http://zeri.info/sport/) and then it will go to crawl each page of their archive (hundreds of thousands of pages) but it won't crawl the links inside these pages. Thanks a lot!
Technical SEO | | gjergjshala0 -
E-commerce website
Hi, I have to do a SEO optimization on a huge e-commerce website, but i don´t know if i have to focus in Schema or meta tags ( page title, meta description) which of them is more important? how can I optimize the website? Thanks
Technical SEO | | AbacoDigital1 -
Hreflang and canonical for multi-language website
Hi all, We're about to have a new website in different languages and locations, which will replace the existing one. Lets say the domain name is example.com. the US version will be example.com/en-us/ and the UK version will be example.com/en-uk/. Some of the pages on both version share the same content. So in order to solve it, we're about to use hreflang on each page + a canonical tag which will always use the US address as canonical address. My question is - since we are using canonical tag along with hreflang, is there a possibility that a user who is searching with Google.co.uk will get the canonical US address instead of the UK address? Or maybe the search engine will know to display the right localized address since (UK) i've been using hreflang? It is really important for me to know, because i'm afraid we will lose the high rankings that we have right now on google.co.uk. Thanks in Advance
Technical SEO | | Operad-LTD0 -
Server 500: website deindexed?
Hi mozzers, Since August 22nd, (not a site I manage) has had a Server error 500 and all the pages got deindexed? This is obviously a server issue but why it got deindexed is it because it's been a while since it had this server issue? On the pages I checked the pages loads correctly so I am a bit confused here! His webmaster account show 1500 server errors! Can someone tell me what is going on and how to fix it? Thanks
Technical SEO | | Ideas-Money-Art0 -
Google not indexing my website
Hi guys, We have this website http://www.m-health-expo.nl/ but it is not indexed by google. In webmaster tools google says that it can not fetch the site due to the robots.txt but i do not see any faults in it. http://www.m-health-expo.nl/robots.txt Do you see something strange, it really bothers me.
Technical SEO | | RuudHeijnen0 -
Resurrecting a heavily duplicated Joomla website.
The site in question is stanwix.info. I set this up years ago as a community project before I had even heard of seo. It carries quite a bit of content but has appallingly "SEO Unfriendly" URLs some content is recorded as duplicate 3 or more times because of these dynamic URLs according to the crawl statistics returned by SEO Moz. Currently it ranks well for the keyword stanwix - not too surprising as the only significant local competition is a large commercial holiday complex located several miles away out on the coast. As a sort of showcase, I have been trying to come up with a strategy to re-optimise stanwix.info so that it can outdo the commercial site stanwix.com I am pretty sure that step 1 involcves upgrading from the current archaic version of Joomla to the latest release even though that will break some of the modules currently installed. My pal who actually coded the site did a fair bit of hacking of the modules to get them to work properly. OR do I simply bin it and start a fresh site consigning years worth of community content to the darkness and replace with fresh material - I'm not keen on this option as I have put a lot of heart and soul into this project. I am in a quandary here as the problems are so numerous that whatever I do will break the site for some time. The reason I want to keep the site is because the business directory provides a useful locally relevant source of links for other clients and I am also going to set up a separate website for the community centre on a subdomain - this time running in wordpress which I do have a fair bit of experience with. Your advice for my addled brain would be much appreciated.
Technical SEO | | SkiBum0
