Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
-
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep.
I just wanted to know the possible reasons and has anyone encountered this before?
-
Hi Moosa, do you have an update on this issue?
-
Things I check:
- header check (I've had pages appear 200 OK in webconfs and then massive errors in Browseo so I often check both.)
- Screaming Frog check - does the direct URL start a proper crawl?
- Make sure it's not blocked in WMT under removed pages (some website owners and/or bad SEOs try to demote sitelinks by removing the URL.)
- Meta tags & robots.txt both of which I assume you already checked.
- Duplicate content being indexed. Take a random sentence & throw it in quotes on Google. If a 100% dupe version comes up on their facebook page, archive.is, or some directory it may just need a content refresh to push through.
If you check all 5 of those off, then I'd definitely like to see the URL.
-
If that doesn't work, please share the page URL.
-
Thanks Drik,
I have double check the accessibility of page to crawlers and bots and pages does have decent amount of internal and external links pointing to the page.
Thanks for the Google fetch idea, i am going to try it to see if it works and update you accordingly.
-
Hi
I guess you have already checked that the page is accessible for crawlers and has sufficient internal links to be discovered by the bots (if not - try crawling the site with Screaming Frog)
If it's only one page that isn't picked up you could try a fetch like google & submit to index to force the page being indexed.
It could be that Google considers this page as less important and crawls with lower frequency compared to the other pages (to deep in the site, to few links, not enough content,...)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Combining products - edit existing product page or 301 redirect to new page?
We want to combine existing products - e.g. 'hand lotion' and 'body lotion' will become 'hand & body lotion'. As such, we'll need to combine the two product pages into one. What would be the best route to take in terms of SEO to do this? My initial reaction is to create a new product page and then 301 or 302 redirect the old products to the new product page depending on if the change is permanent or temporary. Would you agree? Or am I missing something?
On-Page Optimization | | SwankyApple1 -
Does Google avoid indexing pages that include registered trademark signs?
I am suspecting that Google often hesitates to index pages that have registered trademarks on them that are marked with a ®. For example EGOL® used in the title tag or in the tag at the top of the page. Registered trademarks are everywhere and most retail product pages contain at least one of them. However, most people use the registered trademark names as text in their writing without adding the registered trademark sign of ®. Have you experienced a problem getting such pages indexed or have you read any articles about how Google treats registered trademarks?
On-Page Optimization | | EGOL0 -
Do Google's mobile friendly updates effect visibility on desktop results pages?
Google say that their quest to make websites more mobile friendly impacts mobile search results - https://webmasters.googleblog.com/2016/03/continuing-to-make-web-more-mobile.html But I am wondering if having a website that is less mobile friendly effects desktop SERPs as well? We require Adobe Flash as a tool for people to upload their images to us but not on the landing pages we're trying to rank. So our landing pages are not as mobile friendly as they could be (which we're looking to improve) but am worried this is effecting desktop search results even though Google do not claim they do.
On-Page Optimization | | KerryK1 -
Should I set paramters to maximize the number of producs on all category pages for a retail website?
We have a retail website with over 20K products and do many things to reduce duplicate content including canonicals & parameters. In setting up parameters in Google Search Console, we have told Google to ignore the option to list more products, so the category pages that are indexed have only the default 12 products on them. I don’t want to create too much duplicate product content by doing this, but if there are only 12 products on a category page, the content is very thin Is it better to have as much product content as possible? My question: Should we set the parameter to follow the default (12 products) or should set it to 200 so there are more products associated with the category page?
On-Page Optimization | | apsltd1 -
Should a website have the same standard headings for product pages? And how does that affect SEO?
Hello! I hope my question is self-explanatory but let me give you an example. Suppose I have website X selling A, B and C and I have a dedicated page for each of those products. For SEO practises, Is it better to have the same webpage structure for all product pages (i.e. the same headings and sections) or is it better to vary the structure of the page? Or is it really not important? Again going back to the example, is it better for product page A, B and C to have the same sections in the same order (i.e. Pricing, Product Details, Product Reviews) or is it better to keep it as a flowing content page (product page A has same information in B and C but all those pages are different with different headings and questions asked depending on the product)? Many thanks!
On-Page Optimization | | MarketingDude0 -
I'm looking to put a quite length FAQs tab on product pages on an ecommerce site. Am I likely to have duplicate content issues?
On an ecommerce site we have unique content on the product pages (i.e. descriptions), as well as the usual delivery and returns tabs for customer convenience. From this we haven't had any duplicate content issues or warnings, which seems to be the case industry-wide. However, we're looking to add a more lengthy FAQs tab which is still highly relevant to the customer but contains a lot more text than the other tabs. The product descriptions are also relatively small. Do you think this will cause potential duplicate content issues or should it be treated the same as a delivery tab, for instance?
On-Page Optimization | | creativemay0 -
SERP listing of a websites' 'categories'
Hi all, just wondering if anyone has thoughts on what I can do to encourage SERP listings that include website categories, eg http://www.google.com.au/search?q=seomoz&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a . I'm assuming search engines only display type of listings when the search query closely matches the domain name? Thank heaps!
On-Page Optimization | | TheWebSearchMarketingCompany0 -
Do we need to use the canonical tag on non-indexed pages?
Hi there I have been working in / learning SEO for just over a year, coming from a non dev background, so there are still plenty of the finer points on-page points I am working on. Slowly building up confidence and knowledge with the great SEOMoz as a reference! We are working on this site http://www.preciseuk.co.uk (we are still tweaking the tags and content by the way- not finished yet!) Because a lot of the information is within accordians, a page is generated for each tab of the accordian expanded, for example: http://www.preciseuk.co.uk/facilities-management.php is the main page but then you also have: http://www.preciseuk.co.uk/facilities-management.php?tab=0 http://www.preciseuk.co.uk/facilities-management.php?tab=1 http://www.preciseuk.co.uk/facilities-management.php?tab=2 http://www.preciseuk.co.uk/facilities-management.php?tab=3 http://www.preciseuk.co.uk/facilities-management.php?tab=4 http://www.preciseuk.co.uk/facilities-management.php?tab=5 All of which are in the same file. According to the crawl test, these pages are not indexed. Because it is all in one file, should we add the canonical tag to it, so that this is replicated in all the tab pages that are generated? eg. Thanks in advance for your help! Liz OneResult
On-Page Optimization | | oneresult
liz@oneresult.co.uk2