Crawling issue
-
Hello,
I am working on 3 weeks old new Magento website. On GWT, under index status >advanced, I can only see 1 crawl on the 4th day of launching and I don't see any numbers for indexed or blocked status.
| Total indexed | Ever crawled | Blocked by robots | Removed |
| 0 | 1 | 0 | 0 |I can see the traffic on Google Analytic and i can see the website on SERPS when i search for some of the keywords, i can see the links appear on Google but i don't see any numbers on GWT.. As far as I check there is no 'no index' or robot block issue but Google doesn't crawl the website for some reason.
Any ideas why i cannot see any numbers for indexed or crawled status on GWT?
Thanks
Seda
| | | | |
| | | | | -
Thanks Davenport and Everett, I've got XML sitemap submitted already, checked robot and no index etc but no stats yet. I'll wait for a few weeks more but it just doesn't make sense to not get any stays after a month. Meanwhile, If i figure out anything, I'll reply here.
-
The data in GWT is not always updated regularly. Also, for a new site that has never been indexed before and has no, or few, external links, it would not be surprising to experience infrequent crawls. The more links you earn and the more of a history of fresh content and updated pages you develop, the more often and deeply you'll be crawled.
As Davenport-Tractor mentioned, an XML sitemap submitted to GWT will also help if you haven't done that already.
If most of your pages are indexed when you do a (site:yourdomain.com) search on Google I wouldn't worry about it too much. If they aren't indexed, you may have a problem, such as inadvertently blocking the crawlers via robots meta tag or robots.txt file. I'd have to see the site to know that though.
-
Seda,
Have you submitted a sitemap to GWMT?
That will greatly help the Google spiders crawl your site. Kind of like telling someone how to find your business vs providing them a road map. They will get there a whole lot quicker if you provide a map on how to find all the different locations.
There are quite a few different sitemap generator programs available. These programs will index your site and build the sitemap.xml file for you. Now you can save the file to your website root directory, then point GWMT to the sitemap.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Canonical Issue On AMP
Hi everyone,
Intermediate & Advanced SEO | | MuhammadQasimAttari
I have one issue about canonical. kindly guide me about it. I have a site example.com/abc and I convert it on an amp and know its URLs is example.com/abc=?amp. but the search console tells me to add the proper canonical URL but both pages are the same. kindly guide me about it. what will I do?0 -
Issue with Google Structured Data Testing Toll asking for "logo" - ld+json
Hi I am trying to get schema set up for a number of articles we are putting on our site (eg:https://www.plasticpipeshop.co.uk/temporary-KB-page_ep_88-1.html) the mark up I think I should use is : Google structured data testing tool keeps insisting I have "publisher" and then "logo" but doesn't seem to want accept anything for the "logo" entry no matter how I seem to code it. Any assistance would be much appreciated as after three hours on this I am pulling what little hair I have left out! Bob
Intermediate & Advanced SEO | | BobBawden10 -
How to solve this issue and avoid duplicated content?
My marketing team would like to serve up 3 pages of similar content; www.example.com/one, www.example.com/two and www.example.com/three; however the challenge here is, they'd like to have only one page whith three different titles and images based on the user's entry point (one, two, or three). To avoid duplicated pages, how would suggest this best be handled?
Intermediate & Advanced SEO | | JoelHer0 -
Sever SERP Issues on Teem.com (Long Post)
I work for Teem.com. Here is the story:We used to be Eventboard.io and we enjoyed strong rankings and a healthy organic presence. We changed our name, our website, and expanded upon on our product offerings and launched Teem.com. A lot of content was very similar from Eventboard.io. We had 11 of our big keywords ranking in the top 3 positions. We launched the new website in early October of 2016.Here is what we have done: We setup a network of 301 redirects for the homepage, company pages, events page, blog posts, and each and every long tail page. Everything from the old site had a new place to live on the new site with very similar content. This list was then passed to our server/it folk to implement (we run a StaticPress site so we don't control those from within WordPress). Both the old and new site are WordPress websites. Setup a site domain move through Google Search Console Combed through Teem.com to take care of SEO issues using various tools. We know there are still some issues (speed, etc.) that aren't helping us, but we are in a good state overall in terms of technical SEO. Deep dive into the domain name history, backlinks, internal linking (which could be better). Developed more long-tail content (more coming). Here is what is weird: We have almost no organic traffic (or traction) since our rebrand. We understood we would be hit hard as the domain name was changed, the content changed, and the CMS was revamped. The only real organic search traffic we get is branded to our old name (which is luckily the name of one of our products): Eventboard. We rank well for this and see high conversion from this keyword. We rank very well for "conference room displays" on Bing for our long tail and home page, but we show up at position 23 for our iTunes app page on Google and 33 for the long tail page. We dominate in bing for our company name "Teem" and finally show up Google for our Facebook page in position 13th. Our website is way way down the list (beyond page 5) for the exact company name with super low competition. Site performance has been good, user feedback has been good, site uptime has been great. No red flags here. No blaring errors in search console besides maybe a few 404 pages that are cleaned up every few weeks. We have no idea what to do. Have engaged with multiple SEO agencies. Been told over and over to be patient because of the changes we have made, but we still see no progress 6 months later.We think the issue might be related to something misfiring with our 301 redirects, based on some referral information.Any insight would be greatly, greatly appreciated. We are stumped. Thanks for any help!
Intermediate & Advanced SEO | | brycedmorgan0 -
Is legacy duplicate content an issue?
I am looking for some proof, or at least evidence to whether or not sites are being hurt by duplicate content. The situation is, that there were 4 content rich newspaper/magazine style sites that were basically just reskins of each other. [ a tactic used under a previous regime 😉 ] The least busy of the sites has since been discontinued & 301d to one of the others, but the traffic was so low on the discontinued site as to be lost in noise, so it is unclear if that was any benefit. Now for the last ~2 years all the sites have had unique content going up, but there are still the archives of articles that are on all 3 remaining sites, now I would like to know whether to redirect, remove or rewrite the content, but it is a big decision - the number of duplicate articles? 263,114 ! Is there a chance this is hurting one or more of the sites? Is there anyway to prove it, short of actually doing the work?
Intermediate & Advanced SEO | | Fammy0 -
Best way to block a search engine from crawling a link?
If we have one page on our site that is is only linked to by one other page, what is the best way to block crawler access to that page? I know we could set the link to "nofollow" and that would prevent the crawler from passing any authority, and we can set the page to "noindex" to prevent it from appearing in search results, but what is the best way to prevent the crawler from accessing that one link?
Intermediate & Advanced SEO | | nicole.healthline0 -
Concerns about duplicate content issues with australian and us version of website
My company has an ecommerce website that's been online for about 5 years. The url is www.betterbraces.com. We're getting ready to launch an australian version of the website and the url will be www.betterbraces.com.au. The australian website will have the same look as the US website and will contain about 200 of the same products that are featured on the US website. The only major difference between the two websites is the price that is charged for the products. The australian website will be hosted on the same server as the US website. To ensure Australians don't purchase from the US site we are going to have a geo redirect in place that sends anyone with a AU ip address to the australian website. I am concerned that the australian website is going to have duplicate content issues. However, I'm not sure if the fact that the domains are so similar coupled with the redirect will help the search engines understand that these sites are related. I would appreciate any recommendations on how to handle this situation to ensure oue rankings in the search engines aren't penalized. Thanks in advance for your help. Alison French
Intermediate & Advanced SEO | | djo-2836690 -
Will the Canonical tag fix this issue?
I recently joined promoz and I've been busy working through the issues raised brought to light during the crawls of our Magento site, www.unitedbmwonline.com. One of many issues were the 10,000+ Duplicate Page Titles which I believe are the result of not using Canonical tags when setting up the store. This is now corrected and hopefully I'll see a significant drop in this value after this next crawl. Am I correct in this assumption? Cheers, Steve
Intermediate & Advanced SEO | | SteveMaguire0