Crawling issue
-
Hello,
I am working on 3 weeks old new Magento website. On GWT, under index status >advanced, I can only see 1 crawl on the 4th day of launching and I don't see any numbers for indexed or blocked status.
| Total indexed | Ever crawled | Blocked by robots | Removed |
| 0 | 1 | 0 | 0 |I can see the traffic on Google Analytic and i can see the website on SERPS when i search for some of the keywords, i can see the links appear on Google but i don't see any numbers on GWT.. As far as I check there is no 'no index' or robot block issue but Google doesn't crawl the website for some reason.
Any ideas why i cannot see any numbers for indexed or crawled status on GWT?
Thanks
Seda
| | | | |
| | | | | -
Thanks Davenport and Everett, I've got XML sitemap submitted already, checked robot and no index etc but no stats yet. I'll wait for a few weeks more but it just doesn't make sense to not get any stays after a month. Meanwhile, If i figure out anything, I'll reply here.
-
The data in GWT is not always updated regularly. Also, for a new site that has never been indexed before and has no, or few, external links, it would not be surprising to experience infrequent crawls. The more links you earn and the more of a history of fresh content and updated pages you develop, the more often and deeply you'll be crawled.
As Davenport-Tractor mentioned, an XML sitemap submitted to GWT will also help if you haven't done that already.
If most of your pages are indexed when you do a (site:yourdomain.com) search on Google I wouldn't worry about it too much. If they aren't indexed, you may have a problem, such as inadvertently blocking the crawlers via robots meta tag or robots.txt file. I'd have to see the site to know that though.
-
Seda,
Have you submitted a sitemap to GWMT?
That will greatly help the Google spiders crawl your site. Kind of like telling someone how to find your business vs providing them a road map. They will get there a whole lot quicker if you provide a map on how to find all the different locations.
There are quite a few different sitemap generator programs available. These programs will index your site and build the sitemap.xml file for you. Now you can save the file to your website root directory, then point GWMT to the sitemap.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Interest in optimise Google Crawl
Hello, I have an ecommerce site with all pages crawled and indexed by Google. But I have some pages with multiple urls like : www.sitename.com/product-name.html and www.sitename.com/category/product-name.html There is a canonical on all these pages linking to the simplest url (so Google index only one page). So the multiple pages are not indexed, but Google still comes crawling them. My question is : Did I have any interest in avoiding Google to crawl these pages or not ? My point is that Google crawl around 1500 pages a day on my site, but there are only 800 real pages and they are all indexed on Google. There is no particular issue, so is it interesting to make it change ? Thanks
Intermediate & Advanced SEO | | onibi290 -
Issues with Sitelinks
Hey Everyone, I found a couple threads where this was asked, but unfortunately I haven't seen any responses that explain it yet. My issue is with the sitelinks Google is choosing for a section of my website; they're using wrong/lowercase anchor text for a couple of the page titles that isn't anywhere on the page or in the backend that I can find (see screenshot). Any thoughts/reasons as to why they'd be using the lowercase text there? Thanks in advance! David 9r9Pz4w
Intermediate & Advanced SEO | | davidkaralisjr0 -
Htaccess Issue: URL not resolving properly
I am merging a niche site, tshirts.com to another site mainsite.com. I am using an htaccess file on a linux server, and the homepage of the niche site is being directed to the corresponding category page on the main site (i.e nichesite.com to mainsite.com/niche.html). Everything else is also a page to page redirect. I have something like this in the htaccess file: Redirect 301 http://tshirts.com/ http://www.mainsite.com/tshirts.html
Intermediate & Advanced SEO | | inhouseseo
Redirect 301 http://tshirts.com/blue.html http://www.lampclick.com/blue-t-shirts.html
Redirect 301 http://tshirts.com/white.html http://www.mainsite.com/white-t-shirts.html
Redirect 301 http://tshirts.com/black-tshirts.html http://www.mainsite.com/bk-t-shirts.html When I check 301 for lets say http://tshirts.com/blue.html, I get: http://tshirts.com/blue.html -** 301 Moved Permanently** http://www.mainsite.com/tshirts.htmlblue.html -** 302 Found** http:www.mainsite.com/ How do I fix this? Why is everything being appended to minsite/tshirts.html? I appreciate your help.0 -
Google adwords destination link issue
In google adwords I set destination link like this http://www.abcd.com/ and like this http/:www.abcd.com so how google adwords react on both this? And how it will show data in adwords? Thanks! Dev
Intermediate & Advanced SEO | | devdan0 -
Massive drop in organic rankings - but strange issues
Hi guys, We experienced a massive drop in organic rankings on Saturday 29 October, a drop of around 80%.
Intermediate & Advanced SEO | | cashchampion
I have received no notifications in webmaster tools
The site still ranks for its name so we are not banned
The site has been around since 2007
My mozrank and trustrank are very high (higher than sites that now still rank while we don't)
The strange this is the homepage doesn't rank for the main keyword (which it always did), but now a deeper page ranks for the main keyword, albeit on page 4.
Quite a few of our sites were affected, even though they target different industries, different countries, with different hosts, different unique content, different links (for one we havent even started link building and it was getting 250 UV per day, and now 8). I am completely confused by this.
Any advice or direction would be extremely helpful. Thanks so much0 -
Any Issues with Changing a Page based on IP address?
Building a site and wondering if we have one page that changes depending on where how it is accessed if that is a good / bad idea. Thanks in advance!
Intermediate & Advanced SEO | | nicole.healthline0 -
Will implementing a 'Scroll to Div Anchor' cause a duplicate content issue?
I have just been building a website for a client with pages that contain a lot of text content. To make things easier for site visitors I have created a menu bar that sticks to the top of the page and the page will scroll to different areas of content (i/e different Div id anchors) Having done this I have just had the thought that this might inadvertently introduce duplicate content issue. Does anyone know if adding an #anchor to the end of a url will cause a duplicate content error in google? For example, would the following URLs be treated as different:- http://www.mysite.co.uk/services
Intermediate & Advanced SEO | | AdeLewis
http://www.mysite.co.uk/services#anchor1
http://www.mysite.co.uk/services#anchor2
http://www.mysite.co.uk/services#anchor3
http://www.mysite.co.uk/services#anchor4 Thanks.0 -
Rankings Issue
Hey guys, and gals, So our site http://www.motorcyclecenter.com/ is having the hardest time ranking in the big G. We've built links, optimized, and all the other basics. If you guys have a chance, could you take a glance, and tell us what you think, as to why this site is having such a hard time. Much appreciated!
Intermediate & Advanced SEO | | leatherupseo0