Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
-
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep.
I just wanted to know the possible reasons and has anyone encountered this before?
-
Hi Moosa, do you have an update on this issue?
-
Things I check:
- header check (I've had pages appear 200 OK in webconfs and then massive errors in Browseo so I often check both.)
- Screaming Frog check - does the direct URL start a proper crawl?
- Make sure it's not blocked in WMT under removed pages (some website owners and/or bad SEOs try to demote sitelinks by removing the URL.)
- Meta tags & robots.txt both of which I assume you already checked.
- Duplicate content being indexed. Take a random sentence & throw it in quotes on Google. If a 100% dupe version comes up on their facebook page, archive.is, or some directory it may just need a content refresh to push through.
If you check all 5 of those off, then I'd definitely like to see the URL.
-
If that doesn't work, please share the page URL.
-
Thanks Drik,
I have double check the accessibility of page to crawlers and bots and pages does have decent amount of internal and external links pointing to the page.
Thanks for the Google fetch idea, i am going to try it to see if it works and update you accordingly.
-
Hi
I guess you have already checked that the page is accessible for crawlers and has sufficient internal links to be discovered by the bots (if not - try crawling the site with Screaming Frog)
If it's only one page that isn't picked up you could try a fetch like google & submit to index to force the page being indexed.
It could be that Google considers this page as less important and crawls with lower frequency compared to the other pages (to deep in the site, to few links, not enough content,...)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can service request pages be indexed for a service site?
I think there is no point in indexing service request pages for a service site. And it causes the indexing of the main pages to be done with a delay. Does anyone have experience with indexing service request pages and their results?
On-Page Optimization | | sora.ya04680 -
How do I redirect SEO from pages on old website to new website with new domain name?
How do I redirect SEO from pages on old website to new website with new domain name? My website in squarespace has good rankings and we have rebranded to a new name and enw domain name, how do I move across the rankings fomr the old site in squarespace to my new site thats in wordpress please?
On-Page Optimization | | Carrotpower0 -
Is it OK to put a Blog Post and a Page within the same folder on a Wordpress hosted website?
Our education company website (hosted on wordpress) has evolved into having content on key topics distributed across both blog posts and pages. For example, "top-pharmaceutical-companies" lends itself to being published as a page. However other content "top-pharmaceutical-companies-usa-2016" lends itself to being published as a blog post as it's more temporal in nature. Now we'd like to establish topical domain relevance for the root keyword "pharmaceutical companies" and build a folder www./ourcompany/pharmaceutical-companies/ But when we look through our blog content, we notice we have "Blog Posts" that would be an excellent fit for certain folders within our "Page" url structure. So would it be OK to amend these blogs post urls addresses to place them within the folder structure of the pages.
On-Page Optimization | | GetReskilled0 -
Is there a limit to the number of duplicate pages pointing to a rel='canonical ' primary?
We have a situation on twiends where a number of our 'dead' user pages have generated links for us over the years. Our options are to 404 them, 301 them to the home page, or just serve back the home page with a canonical tag. We've been 404'ing them for years, but i understand that we lose all the link juice from doing this. Correct me if I'm wrong? Our next plan would be to 301 them to the home page. Probably the best solution but our concern is if a user page is only temporarily down (under review, etc) it could be permanently removed from the index, or at least cached for a very long time. A final plan is to just serve back the home page on the old URL, with a canonical tag pointing to the home page URL. This is quick, retains most of the link juice, and allows the URL to become active again in future. The problem is that there could be 100,000's of these. Q1) Is it a problem to have 100,000 URLs pointing to a primary with a rel=canonical tag? (Problem for Google?) Q2) How long does it take a canonical duplicate page to become unique in the index again if the tag is removed? Will google recrawl it and add it back into the index? Do we need to use WMT to speed this process up? Thanks
On-Page Optimization | | dsumter0 -
Rel canonical tag on a single page site?
I have a wordpress theme site which essentially is all in 1 page. Do I need to use rel-canonical tag? It would just loop?
On-Page Optimization | | graftene0 -
Can you 301 redirect to a page that has other pages 301 to it?
Two years ago updated url page to include better keywords and used a 301 redirect from the old page to the new. so www.example.com/keyword-1st-generation.html now points to ... www.example.com/keyword-2nd-generation.html That moved the pages up in ranking, but now have better kw for the url, so is it okay to redirect the /keyword-2nd-geration-html to www.example.com/keyword-3rd-generation.html And what is a good length of time before removing the 1st-generation url? It's been 3 years and there is no chance of using it again. Plus, no sign of it in analytics.
On-Page Optimization | | AllIsWell0 -
What precaution should we take to change the default page of the site
For some reason we wanted to change the default page of my site from example.com to example.com/default.aspx. We will be using 301 redirection to get the back link benefits. Do we need to make any changes in webmaster tool and sitemap too??
On-Page Optimization | | CyrilWilson0 -
Does a page's url have any weight in Google rankings?
I'm sure this question must have been asked before but I can't find it. I'm assuming that the title tag is far more important than the page's url. Is that correct? Does the url have any relevance to Google?
On-Page Optimization | | rdreich490