Website not crawled
-
i added website www.nsale.in in add campaign, it shows only 1 page crawled. but its working fine for other sites, any idea why it failed ?
-
This page references several reasons why only one page may be crawled. If it isn't any of these reasons, please contact our help desk. Thanks!
-
just looked at the other site you say is working... it has a much simpler robots file and no delay. just sayin' //
-
That's one of the bigger robots files I've seen in awhile..
You sure that crawl delay isn't messing things up? Not sure but that'd be my first guess.
-
its from day one. i have other domains on the same server www.litetab.in, its crawling properly. not sure why www.nsale.in alone doesnt work.
-
How long has it been up? Also, did you add the site from its root?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
A crawl revealed two home pages
After doing a site crawl using the moz tool, I have found two home pages-www.domain.com/ and www.domain.com. Both URLS have the exact same metrics and I have set a preferred domain name in google, will this hurt seo? Should I claim the www.domain.com/ as well as www.domain.com and domain.com in the search console? Thanks
Technical SEO | | Tom3_150 -
Mobile website content optimisation
Hi there, someone I know is going to put their site to a mobile version with a mobile sub domain (m.). I have recommended responsive but for now this is their only way forward to cope with the 21st April update by Google. My question is what is the best practice for content, as its a different url will there need to be a canonical tag in to stop duplication and thus being penalised from the Google panda update? Any advice much appreciated.
Technical SEO | | tdigital0 -
Website Revision
Hi all~ We are completely remaking our website: www.containmydog.com. I believe we have a good handle on the visual aspects of the redesign. What are the backend or behind the scenes (probably not using the technical term) things that need to be done so search engines know where things are. For example i know we are not going to remove some pages , change were some pages are on the site and add new pages. Is there a checklist that lists the important things to do when designing/redesigning a website? If there is not a checklist what are the things I should be asking the web person we hire?
Technical SEO | | PhotographerSteve1 -
Server 500: website deindexed?
Hi mozzers, Since August 22nd, (not a site I manage) has had a Server error 500 and all the pages got deindexed? This is obviously a server issue but why it got deindexed is it because it's been a while since it had this server issue? On the pages I checked the pages loads correctly so I am a bit confused here! His webmaster account show 1500 server errors! Can someone tell me what is going on and how to fix it? Thanks
Technical SEO | | Ideas-Money-Art0 -
Crawling issues in google
Hi everyone, I think i have crawling issues with one of my sites. It has vanished form Google rankings it used to rank for all services i offered now it doesn't anymore ever since September 29th. I have resubmitted to Google 2 times and they came back with the same answer: " We reviewed your site and found no manual actions by the web spam team that might affect your site's ranking in Google. There's no need to file a reconsideration request for your site, because any ranking issues you may be experiencing are not related to a manual action taken by the webspam team. Of course, there may be other issues with your site that affect your site's ranking. Google's computers determine the order of our search results using a series of formulas known as algorithms. We make hundreds of changes to our search algorithms each year, and we employ more than 200 different signals when ranking pages. As our algorithms change and as the web (including your site) changes, some fluctuation in ranking can happen as we make updates to present the best results to our users. If you've experienced a change in ranking which you suspect may be more than a simple algorithm change, there are other things you may want to investigate as possible causes, such as a major change to your site's content, content management system, or server architecture. For example, a site may not rank well if your server stops serving pages to Googlebot, or if you've changed the URLs for a large portion of your site's pages. This article has a list of other potential reasons your site may not be doing well in search. " How i detected that it may be a crawling issue is that 2 weeks ago i changed metas - metas are very slow in getting updated and for some of my pages never did update Do you know any good tools to check for bad code that could slow down the crawling. I really don't know where to look other than issues for crawling. I validated the website with w3c validator and ran xenu and cleaned these up but my website is still down. Any ideas are appreciated.
Technical SEO | | CMTM0 -
I have 2 websites with the same content
Hello everyone, this is my first post here on SEOmoz and I have a questions that I cannot seem to figure out. So here is my scenario: I have 2 websites that are identical. The only difference between the 2 websites is the domain name. This was done a while back for marketing purposes, however, I am no longer needing my 2nd website. What is the best way to get rid of this second website? I still have about 1 paying customer a day convert on this 2nd website and I do not want to loose them, however, I know that I am getting penalized by the search engines because of this duplicate content. Please let me know the best way of going about this. PS: I have read about 301 redirects, canonicalizing URLs, and other methods but do not know which one to choose. Any help is greatly appreciated!
Technical SEO | | threebiz0 -
Do you validate you websites?
Do you consider the guidelines from http://validator.w3.org/ when setting up a new website? As far as I know they don't influence rankings ... What is your opinion about that topix?
Technical SEO | | petrakraft0
