Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
-
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep.
I just wanted to know the possible reasons and has anyone encountered this before?
-
Hi Moosa, do you have an update on this issue?
-
Things I check:
- header check (I've had pages appear 200 OK in webconfs and then massive errors in Browseo so I often check both.)
- Screaming Frog check - does the direct URL start a proper crawl?
- Make sure it's not blocked in WMT under removed pages (some website owners and/or bad SEOs try to demote sitelinks by removing the URL.)
- Meta tags & robots.txt both of which I assume you already checked.
- Duplicate content being indexed. Take a random sentence & throw it in quotes on Google. If a 100% dupe version comes up on their facebook page, archive.is, or some directory it may just need a content refresh to push through.
If you check all 5 of those off, then I'd definitely like to see the URL.
-
If that doesn't work, please share the page URL.
-
Thanks Drik,
I have double check the accessibility of page to crawlers and bots and pages does have decent amount of internal and external links pointing to the page.
Thanks for the Google fetch idea, i am going to try it to see if it works and update you accordingly.
-
Hi
I guess you have already checked that the page is accessible for crawlers and has sufficient internal links to be discovered by the bots (if not - try crawling the site with Screaming Frog)
If it's only one page that isn't picked up you could try a fetch like google & submit to index to force the page being indexed.
It could be that Google considers this page as less important and crawls with lower frequency compared to the other pages (to deep in the site, to few links, not enough content,...)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Making Shopify URL's Simpler - Losing the words 'collection', 'product' and 'page' in a Shopify store URL. Any advice?
Hi Mozers! I have a Shopify store (of which there are many advantages) however one big SEO disadvantage, is that my URL structures contravene all Moz advice on dynamic URL structure and whats more I am reminded about this every week when I have a Moz site crawl and I have a batch of URL's that are longe than the 75 characters. A Shopify URL will run www.domain name.com/collections/collection-name/product/product-name. According to advice a it should be www.domain name.com/collection-name/product-name - Don't even get started on sub-collections! I sell portfolio books, album etc and keepsake memory boxes (so long keywords) AND, I have a long(ish) business name. So, For user experience and keyword length, do I just ignore trying to achieve a dynamic URL under 75 characters? When I have asked Shopify, the say their URL's are an integral part of the "Ruby on Rails" system, so nothing can be done Or can it ??? I can't be the only Moz member with this issue can I ??
On-Page Optimization | | nick_HandCo0 -
Penalty for Changing Home Page Title Tags
Hey Mozzers I'm certain of the answer to this question, however I wanted to get some input from the experts in Moz-land to hopefully provide some additional perspective. I recently disagree with a client's assertion that there is some penalty Google levels for changing the title tags of your home page. Now, I understand changing the title tags can influence serp rankings, however, is anyone aware of some penalty Google levels for simply changing the title tags? Most of what I've read and experienced has people changing them all the time without some phantom penalty. It seems to me a problem of correlation = causality, in that people often attribute a drop to an action that may not have actually been the cause. Anyway, if you have any particular insight on this top I would appreciate it greatly. thanks!
On-Page Optimization | | BrandLabs0 -
Pages indexed but not seen in wordpress
Hi there, the crawler of SEOMOZ has detected that some pages have no title or a short metadescription. The problem is that I can't see these pages in wordpress (in the backend). These pages are irrelevant and must be deleted but I can't find them. How could they be indexed by the Googlebot? Where can I find them? Any hint? Thanks all.
On-Page Optimization | | juanmiguelcr0 -
On-Page Report Card: Whats up with the TITLE of the page?
Started to fix the SEO issues on a customers website. When I run a "On-Page Report Card" It says that the title of the webpage:
On-Page Optimization | | maklarlabbet
www.visbymaklarna.se/visbymaklarna.html Is "visbymäklarna - Ditt förstahandsval på gotland." But if I check in the source code of the webbrowser the title should be:
name="title" content="Vi är mäklarna på Gotland som sätter människan i första rummet" /> (Actually this is with special encoding for the swedish characters. The title in coded text is: "Vi är mäklarna på Gotland som sätter människan i första rummet") Anyway the title of the webpage source code and the title of what SEOmoz reports is different. Why is that?0 -
On page link question, creating an additional 'county' layer between states and zips/cities
Question We have a large site that has a page for all 50 states. Each of these pages has unique content, but following the content has a MASSIVE amount of links for each zip AND city in that state. I am also in the process of creating unique content for each of these cities and zips HOWEVER, I was wondering would it make sense to create an additional 'county' layer between the states and the zips/cities. Would the additional 'depth' of the links bring down the overall rank of the long tail city and zip pages, or would the fact that the counties would knock the on page link count down from a thousand or so, to a management 50-100 substantially improve the overall quality and ranking of the site? To illustrate, currently I have State -> city and zip pages (1200+ links on each state page) what i want to do is do state -> county (5-300 counties on each state page) -> city + zip (maybe 50-100 links on each county page). What do you guys think? Am I incurring some kind of automatic penalty for having 1000+ links on a page?
On-Page Optimization | | ilyaelbert0 -
Please review page for on page SEO
I'm working on an adult dating site review blog and am hoping to build links to the category pages. It's a wordpress blog so the posts are categorised by topics. However can you tell me if there is enough content on the page for google to identify what the page is about? The problem is that the content of each category pages is pulled from the posts. What I think is needed is some static text on the page 2-300 words that use the keyword. Here's an example of a page targeting the keywords 'online dating'. http://www.top-10-dating-reviews.com/category/online-dating/
On-Page Optimization | | SamCUK
[editor's note -- thumbnails on site may not be appropriate for some workplaces] Will the page rank like this if there are a good amount of authorative links or does the page itself need work?0 -
Should I be afraid to launch new website?
I have been redesigning my website in the last 2 months. I'm almost ready to finally launch it. Now I wonder how it will effect my rankings. I have taken a few measures to avoid problems such as: I kept the page names as they are. I kept the structure of the website Optimized the structure of my pages which should benefit my SEO rather then harm it. Yet, if I would experience a drop in rankings, could I regain my rankings by switching to the previous website?
On-Page Optimization | | wellnesswooz0 -
Should H1s be used in the logo? If they are and it is dynamic on each page to relate to the page content, is this detrimental to the site rather than having it in the page content?
On some sites, the H1 is contained within the logo and remains consistent throughout the site (i.e. the company name is in the of the logo). If the h1 in a logo is dynamic for each page (i.e. on the homepage it is company name - homepage) is this better or worse to have it changed out on the logo rather than having it in the page content?
On-Page Optimization | | CabbageTree0