Why is site not being indexed by Google, and not showing on a crawl test??
-
On a site we developed of which .com is forwarded to .net domain, we quit getting crawled by google on about the 20th of Feb. Now when we try to run a crawl test on either url, we get There was an error fetching this page. Error description For some reason the page returned did not describe itself as an html page. It could be possible that the url is serving an image, rss feed, pdf, or xml file of some sort. The crawl tool does not currently report metrics on this type of data. Our other sites are fine and this was up to this date. We took out noodp, noydir today as the only thing we could think of. Site is on WP cms.
-
Site last cached 2nd March
Your site is indexed.
Header's returning 200 codes.
Site can be crawled fine, Xenu finds about 27 pages.
Lynxviewer gets through the page alright.
Only thing I can think of is that robots.txt looks needlessly complicated but should be alright, I would consider stripping it all out and re-running the test, if you get the same error then it's not that, if it is then narrow down what it could be.
If no joy, let me know and I'll have another look.
-
The site is www.innerloophomesreport.net, .com. Thanks.
-
Probably going to need the URL on this one.
I presume you can access the site as a user? What's in your robots.txt file? You using the SEOmoz tools?
-
Hi Robert Fisher,
This problem probably come from the headers of the file and not from the content itself. You might want to look at the headers returned by your URL using one of the following tools :
http://www.seoconsultants.com/tools/headers
http://www.rexswain.com/httpview.html
http://web-sniffer.net/
http://www.g-force.ca/referencement/entetesWhen you got the headers, I suggest you post it here so we can look into it.
Best regards,
Guillaume Voyer.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What things i should do to add more attractiveness in my site?
I have 4 years of experience in feedback and survey of different companies for the people who want to learn some from online education about how to do something? and people want to get ease in daily life skills Kroger feedback provides them with a platform to learn everything about surveys and how to drive some to its highest peaks. It all works for People out there through Education SO I have now my own website ready to go but there is a problem in its on-page ranking factors I shall be thankful to you if some from you answer my question What things I should do to add more attractiveness to my site? take a look at my recent view of my site take a look at this Kroger feedback if somebody from you can answer my request??
Web Design | | SEOTOOLS021 -
DNS Prefetching for wordpress site
We have given below as DNS prefetch in our website. CMS is wordpress. Are these okay? I wonder why fonts.googleapis.com is not working. One of our competitors is using youtube. Can we improve it any how?
Web Design | | vtmoz0 -
ECWID Ecommerce Sites. No Custom URLS?
Is there any way possible to be able to name product urls in website that use ECWID for their ecommerce? They have long and "dirty" urls. For example this running boards site: http://www.runningboards4less.com/general-motors#!/~/product/category=6593890&id=28043027 Isn't this hurting the overall SEO of the site? Especially product pages?
Web Design | | Atlanta-SMO0 -
How to add SEO Content to this site
Hi Great community and hope you guys can help! I have just started on a SEO project for http://bit.ly/clientsite , the clients required initial KPI is Search Engine Rankings at a fairly low budget. The term I use for the site is a "blurb site", the content is thin and the initial strategy I want to employ to get the keyword rankings is to utilize content. The plan is to: add targeted, quality (user experience & useful) and SEO content on the page itself by adding a "read more" link/button to the "blurb" on the right of the page (see pink text in image) when someone clicks on the "read more", a box of content will slide out styled much the same as the blurb itself and appear next to and/or overlay over the blurb and most of the page (see pink rectangle in image) Question: Is this layer of targeted , quality (user experience & useful) and SEO content (which requires an extra click to get to it) going to get the same SEO power/value as if it were displayed traditionally on the initial display? If not, would it be better to create a second page (2<sup>nd</sup> layer) and have the read more link to that and then rel-canonical the blurb to that 2<sup>nd</sup> page, so that all the SEO passes to this expanded content and the second page/layer is what will show up in the rankings? Thanks in advance qvDgZNE
Web Design | | Torean0 -
Why aren't Images in G+ product page posts showing up in SERPs for brand searches?
Before 1-2 weeks ago, our G+ posts containing links to our product pages would show up in in SERPs (when searching for our brand name) with a thumbnail of the product image. Now, they do not (see image below for visual). Our tech team confirmed there hasn't been any coding change that might be to blame and I see that this isn't happening to other sites. Any idea what may be the problem here? tcnhLgy
Web Design | | znotes0 -
Site is losing traffic after relaunch
Hello, We've just relaunched this site in the last several days, and we're seeing some small (but stead) traffic decreases, as well as engagement decreases. We're aware that page speed (about 4 seconds from a non-cached browser) and some 404s are an issue, our team is currently working on both. But we're really looking for some constructive criticism here as to what we need to improve. Other issues to be aware of: lots of our social counts went back to 0s, as lots of URLs changed, and it wasn't possible to migrate comments from the old system, so those have gone back to 0 as well. We wonder if this might be affecting both users and search engines perception of the site. Your input would be greatly appreciated. Thanks,
Web Design | | FishAcct
Paul0 -
How many sites on one hosting account?
How many sites is safe to house on one hosting provider? I use BlueHost and they advertise unlimited domains, but I'm not sure what the negative side effects might be from hosting too many on one hosting service. If it matter at all, I'm using WordPress to build my sites. Pros and Cons?
Web Design | | leafndrop0 -
What is a really great bounce rate for a product or service site? What does Good look like?
I am really curious about a result I have never seen before. Our bounce rate went down a lot on a new site. So, what is good??? Recently, we took on a project with a company that offers a product they install for consumers and who had been in business for 15 plus years. The company is successful, has good customer base of those who have been made very happy, etc. It is not a repeat sale type of product, etc. One and done. Their site when we began talking was roughly a year old and was not well constructed but not terrible. Most of the issues were around I frames, use of older coding, poor SEO, etc. There was not really a way to "redesign" and we built a new site. This became a true collaboration in a B2B environment as the owner pushed us like crazy. Not the bad kind of push, the one that makes you say to your team, "Let's find a way!" The result, IMO, was a gorgeous site. But, as you know, those are a dime a dozen. But, to get to the point, when we took over the account they had a bounce rate of around 45%. I did not see this as either good or bad, but a fact and for this industry probably not bad at all. In all honesty, I was not looking at that as a first metric I wanted to move, but it was obviously at or near the top for all the reasons we know. So, this site is a local business, not an everyday product and gets about 2500 to 3000 uniques per month. If we compare to May of 2011/2012: 2011 2012 Total Visitors 1852 3,298 Uniques 1609 2,740 Pageviews 5,634 23,203 Pages/visit 3.04 7.04 Avg Duration 2.05 3.20 Yes, I am leaving off what we are getting, yes, I am leaving off the site. Please don't hate me. I am really wanting to see what others see with site changes and bounce rates first and will disclose. So, what's a great bounce rate? How do you know?
Web Design | | RobertFisher0