Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
-
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep.
I just wanted to know the possible reasons and has anyone encountered this before?
-
Hi Moosa, do you have an update on this issue?
-
Things I check:
- header check (I've had pages appear 200 OK in webconfs and then massive errors in Browseo so I often check both.)
- Screaming Frog check - does the direct URL start a proper crawl?
- Make sure it's not blocked in WMT under removed pages (some website owners and/or bad SEOs try to demote sitelinks by removing the URL.)
- Meta tags & robots.txt both of which I assume you already checked.
- Duplicate content being indexed. Take a random sentence & throw it in quotes on Google. If a 100% dupe version comes up on their facebook page, archive.is, or some directory it may just need a content refresh to push through.
If you check all 5 of those off, then I'd definitely like to see the URL.
-
If that doesn't work, please share the page URL.
-
Thanks Drik,
I have double check the accessibility of page to crawlers and bots and pages does have decent amount of internal and external links pointing to the page.
Thanks for the Google fetch idea, i am going to try it to see if it works and update you accordingly.
-
Hi
I guess you have already checked that the page is accessible for crawlers and has sufficient internal links to be discovered by the bots (if not - try crawling the site with Screaming Frog)
If it's only one page that isn't picked up you could try a fetch like google & submit to index to force the page being indexed.
It could be that Google considers this page as less important and crawls with lower frequency compared to the other pages (to deep in the site, to few links, not enough content,...)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should Title Tags Differ From H1's?
I never thought this was an issue, but now, I'm not so sure. Is it a problem for your title tags to be identical to your H1's? From Moz' OnPage Grader section on "H1's and Title Tags," I thought they were supposed to both match your keyword. Any thoughts? Thanks, Ruben
On-Page Optimization | | KempRugeLawGroup0 -
What's the best Magento Community blog extension?
We are looking at FishPig's Word Press Integrations extension. has anybody used it? Possibly a dumb question, but is SEO adversely affected by the fact it's a WordPress extension on a Magento site?
On-Page Optimization | | Anne_Marie_English0 -
Why will a Page not rank or improve on a website?
My website is performing well, but I have a couple of popular Brand pages that no matter what I do will not rank organically, they are being indexed, but will just not perform or improve. The rest of the pages throughout the website all seem to be fine, can anyone advise if they have experienced a similar problem or advise why this would happen or even suggest what I should do to fix the problem, we have even gone as far to create a new page and delete the old one, and it still hasn't worked. I have used various tools to test the page and variations of search terms and although it is appearing in Google it is not ranked, although it is ranking and visible in Yahoo and Bing. Any help and advise would be appreciated. Jacqui
On-Page Optimization | | EurekaSolutions0 -
I want to check which pages have been crawled
I would like to find out which pages have been crawled by seomoz on my site
On-Page Optimization | | seoworx1230 -
Why isn't our site being shown on the first page of Google for a query using the exact domain, when its pages are indeed indexed by Google
When I type our domain.com as a query into Google, I only see one of our pages on the homepage, and it's in 4th position. It seems though, that all pages of the site are indexed by google when I type in the query "site:domain.com". There was an issue at the site launch, where the robots.txt file was left active for around two weeks. Would this have been responsible for the fact that another domain ranks #1 when we type in our own domain? It has been around a couple of months now since the site was launched. Thanks in advance.
On-Page Optimization | | featherseo0 -
404 errors on page urls that don't even exist
The Seomoz crawler found 404error of pages dont even exist. Ho can that be possible?? Pages like: URL: http://www.yoxo.it/catalog/seo_sitemap/category/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/
On-Page Optimization | | yoxo0 -
Can't canonical, but need pages to show in Google News
We are a news media site in which much of our content is third-party, and already published by several other sources. Our current version of our CMS doesn't expose head tags, so I can't canonical to the original and avoid a duplicate content penalty. Is it ok for news sites NOT to use canonical, or do I have to NOINDEX until our CMS is fixed?
On-Page Optimization | | Aggie0 -
How do I avoid duplicate content and page title errors when using a single CMS for a website
I am currently hosting a client site on a CMS with both a Canadian and USA version of the website. We have the .com as the primary domain and the .ca is re-directed from the registrar to the Canadian home page. The problem I am having is that my campaign produces errors for duplicate page content and duplicate page titles. Is there a way to setup the two versions on the CMS so that these errors do not get produced? My concern is getting penalized from search engines. Appreciate any help. Mark Palmer
On-Page Optimization | | kpreneur0