Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
-
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep.
I just wanted to know the possible reasons and has anyone encountered this before?
-
Hi Moosa, do you have an update on this issue?
-
Things I check:
- header check (I've had pages appear 200 OK in webconfs and then massive errors in Browseo so I often check both.)
- Screaming Frog check - does the direct URL start a proper crawl?
- Make sure it's not blocked in WMT under removed pages (some website owners and/or bad SEOs try to demote sitelinks by removing the URL.)
- Meta tags & robots.txt both of which I assume you already checked.
- Duplicate content being indexed. Take a random sentence & throw it in quotes on Google. If a 100% dupe version comes up on their facebook page, archive.is, or some directory it may just need a content refresh to push through.
If you check all 5 of those off, then I'd definitely like to see the URL.
-
If that doesn't work, please share the page URL.
-
Thanks Drik,
I have double check the accessibility of page to crawlers and bots and pages does have decent amount of internal and external links pointing to the page.
Thanks for the Google fetch idea, i am going to try it to see if it works and update you accordingly.
-
Hi
I guess you have already checked that the page is accessible for crawlers and has sufficient internal links to be discovered by the bots (if not - try crawling the site with Screaming Frog)
If it's only one page that isn't picked up you could try a fetch like google & submit to index to force the page being indexed.
It could be that Google considers this page as less important and crawls with lower frequency compared to the other pages (to deep in the site, to few links, not enough content,...)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Weird 404 Error URL's - Where to redirect?
For some reason, I end up with a bunch of weird 404 error URLs like these. /KRVWZ/SVfSZ/ /wp-content/upl /MKWTZ/RKYQZ/ /wp-content/upload What should I do with them? Should I redirect them or is there a better way to do something with them. I do use Yoast Premium if that makes a difference. Thanks in advance.
On-Page Optimization | | photoseo10 -
Make Google reindex the website after on-page optimization
Hello Moz Community, I just finished the on-page optimization for a new project and I would like to know how can I make Google to reindex the new link structure, titles and meta tags? Thank you!
On-Page Optimization | | CosminC0 -
Will changing my home page cause rankings to drop?
We are looking at doing a major change with our website. Upper management is wanting to have our home page be our store page, instead of just a landing page. So basically we would be eliminating our current home page, and replacing it with our store page. This is a very large site and our current home page gets a lot of traffic. Issues to deal with will be ranking of the current page content; Internal links to both pages; external links (backlinks) pointing to the two pages, and of course, a potential drop in rankings and traffic. Any ideas on how to best do this? Or not do it at all. 301 redirect? Thanks, Brad
On-Page Optimization | | tdawson090 -
PDF's - Dupe Content
Hi I have some pdfs linked to from a page with little content. Hence thinking best to extract the copy from the pdf and have on-page as body text, and the pdf will still be linked too. Will this count as dupe content ? Or is it best to use a pdf plugin so page opens pdf automatically and hence gives page content that way ? Cheers Dan
On-Page Optimization | | Dan-Lawrence0 -
Will Google Custom Search results on my home page kill it's ranking?
This is probably a dumb question, but here goes anyway. 🙂 On a site I have it would be very useful to the reader to offer a search box that uses a Google Custom Search that I have optimized to search websites that are closely on-topic with my site. I know it sounds bad that I would send people to other sites, but just assume that the reasons are valid for this discussion. My question is, if the search results are set to display on the same page (the home page) as the search box, will the links in the search results to external sites just bleed my page rank to death? I assume it would, but thought I'd check just in case I'm missing something. I have to option to place the results on separate page of my site, and noindex it, but it won't be as powerful as it would be on the home page.
On-Page Optimization | | bizzer0 -
On Page Optimization not updating after SSL change
Last week we changed our URLs to be only HTTPS (they were a mixture of both for a couple of weeks). We did this by adding a 301 redirect from HTTP to HTTPS. However, the On Page Optimization is now showing the same URL/Keyword twice for a few of our pages. One has a grade A, one has a grade B. The grade B complains about "Rel Canonical" not matching, but the other doesn't. This is pretty confusing - should it not have crawled the pages and noticed the 301 redirects and combined the entries or removed the HTTP ones?
On-Page Optimization | | RitchieF19800