Why is my site not getting crawled by google?
-
Hi Moz Community,
I have an escort directory website that is built out of ajax. We basically followed all the recommendations like implementing the escaped fragment code so Google would be able to see the content. Problem is whenever I submit my sitemap on Google webmastertool it always 700 had been submitted and only 12 static pages had been indexed. I did the site query and only a number of pages where indexed.
Does it have anything to do with my site being on HTTPS and not on HTTP?
My site is under HTTPS and all my content is ajax based.
Thanks
-
Sean, What is the URL for your site?
-
Hi There
As Alan mentioned HTML is going to be a much more guaranteed way to get indexed. The HTTPS alone shouldn't be affecting anything. But do you have a different robots.txt for https and http? Is the https one blocking crawlers? Do you have the https version of the site registered in webmaster tools? When you go to crawl stats, how many pages does it show that they are crawling?
-Dan
-
Why build it with Ajax, ajax is good for functionally that needs to load seamlessly but not good for content.
Using escaped fragment seems to not work well as many are having problems getting indexed.Does your content have to be loaded via ajax? why not load it on the page? would be much simpler
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Webmaster tools -Fixing over 20,000+ crawl errors
Hi, I'm trying to gather all the 404 crawl errors on my website after a recent hacking that I've been trying to rectify and clean up. Webmaster tools states that I have over 20 000+ crawl errors. I can only download a sample of 1000 errors. Is there any way to get the full list instead of correcting 1000 errors, marking them as fixed and waiting for the next batch of 1000 errors to be listed in Webmaster tools? The current method is quite timely and I want to take care of all errors in one shot instead of over a course of a month.
Intermediate & Advanced SEO | | FPK0 -
Client is paranoid about Google penguin penalty from getting links from a new website they are building
We have a client that is creating a new promotional website that consists of videos, brands and product reviews (SITE B). After a visitor watches a video on SITE B they will be given a "click to purchase" option that will lead them to the original website (SITE A). Our client is paranoid that since all the outgoing links on the new SITE B are going to the original SITE A there might be algorithm penalty (for one website or both). I find this very unlikely and even recommend "no follow" coding for a peace of mind. However are there any resources/links out there that can back up my argument that they will be alright? Thanks
Intermediate & Advanced SEO | | VanguardCommunications0 -
How to remove an entire site from Google?
Hi people, I have a site with around 2.000 urls indexed in google, and 10 subdomains indexed too, which I want to remove entirely, to set up a new web. Which is the best way to do it? Regards!
Intermediate & Advanced SEO | | SeoExpertos0 -
Author Tag - How do you get your Author Thumbnail on more then one site?
Ive managed to set up the Rel:Author tag on my wordpress site, and the thumbnail is now showing in Google search results Each article I write on my blog has a link to me Author page, which has on it a link to my Google Plus page. So this works great. However I write articles for another few websites and would like to get my author thumbnail image showing for them articles too. How do I do this? Do I need an Author page on the other sites also? I dont think I'll be able to get an author page on the other sites, is it not possible to somehow link direct from the article to my G+ page? How would guest articles work too? Any ideas? Thanks in advance guys
Intermediate & Advanced SEO | | JohnPeters0 -
Got a site in Google news, now what do I do?
I have been working on a site for 7 months, publishing articles each day. The site is dedicated to the niche that I am in. It was accepted in to Google news 1 week ago and I am now getting 3 times the amount of traffic. Its great news but I am now wondering how beneficial it is for me. I sell business management advice and really need to generate leads. Thats how I increase my business Any ideas how I could use this Google news site to do this?
Intermediate & Advanced SEO | | JohnPeters0 -
Link to Google Places, or Google Maps?
On our contact page, we offer a link to view Google Maps for directions. I'm wondering should we be linking to our Google Places page instead, or just stick with the Google Map link? Thanks!
Intermediate & Advanced SEO | | GravitateMCC0 -
How do you prevent the mobile site becoming a duplicate of the full browser site?
We have a larger site with 100k+ pages, we need to create a mobile site which gets indexed in the mobile engines but I am afraid that google bot will consider these pages duplicates of the normal site pages. I know I can block it on the robots.txt but I still need it to be indexed for mobile search engines and I think google has a mobile crawler as well. Feel free to give me any other tips that I should follow while trying to optimize the mobile version. Any help would be appreciated 🙂
Intermediate & Advanced SEO | | pulseseo0 -
Google Freshness Update & Ecommerce Site Strategies
Just curious what other ecommerce SEO's are doing to battle fresh content. We've been having our clients work on internal blogs, adding articles one click away from landing pages, and implement product reviews when possible but I don't know that it's enough. Our bigger customers have landing pages (usually category pages) with very competitive keywords. So my main issue is what to do with fresh content on category pages.. I've toyed with the idea of having the landing page content re written every now and then. We used to use a blog parser to bring snippits of comments from the blog into landing pages but I believe that to be a problem with duplicate content. News snippits from other sites don't seem beneficial either. Anyone have any other ideas?
Intermediate & Advanced SEO | | iAnalyst.com0