Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
-
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep.
I just wanted to know the possible reasons and has anyone encountered this before?
-
Hi Moosa, do you have an update on this issue?
-
Things I check:
- header check (I've had pages appear 200 OK in webconfs and then massive errors in Browseo so I often check both.)
- Screaming Frog check - does the direct URL start a proper crawl?
- Make sure it's not blocked in WMT under removed pages (some website owners and/or bad SEOs try to demote sitelinks by removing the URL.)
- Meta tags & robots.txt both of which I assume you already checked.
- Duplicate content being indexed. Take a random sentence & throw it in quotes on Google. If a 100% dupe version comes up on their facebook page, archive.is, or some directory it may just need a content refresh to push through.
If you check all 5 of those off, then I'd definitely like to see the URL.
-
If that doesn't work, please share the page URL.
-
Thanks Drik,
I have double check the accessibility of page to crawlers and bots and pages does have decent amount of internal and external links pointing to the page.
Thanks for the Google fetch idea, i am going to try it to see if it works and update you accordingly.
-
Hi
I guess you have already checked that the page is accessible for crawlers and has sufficient internal links to be discovered by the bots (if not - try crawling the site with Screaming Frog)
If it's only one page that isn't picked up you could try a fetch like google & submit to index to force the page being indexed.
It could be that Google considers this page as less important and crawls with lower frequency compared to the other pages (to deep in the site, to few links, not enough content,...)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How can I make sure pages with similar content don't damage the other's SEO?
I work for a travel company and I have a 'tour page' targeted for pre-booking and a 'booking pack page' post-booking page, with some similar content but with details such as hostel locations, meeting places and times etc. I want to make sure the tour page keeps the authority as this is what I want to rank on SEO. I've got a couple of similar problems to this across site, there are a few pages on site that are post-sale and don't really need to rank on Google but it would be great if they could contribute to other pages' rankings. Thanks!
On-Page Optimization | | nicolewretham0 -
Hi i have a few pages with duplicate content but we've added canonical urls to them, but i need help understanding what going on
hi google is seeing many of our pages and dupliates but they have canonical url on there https://www.hijabgem.com/index.php/maxi-shirt-dress.html has tags https://www.hijabgem.com/maxi-shirt-dress.html
On-Page Optimization | | hijabgem
has tagshttps://www.hijabgem.com/index.php/quickview/index/view/id/4693
has tags
my question is which page takes authority?and are they setup correct, can you have more than one link rel="canonical" on one page?0 -
To change or not to change. DO we update our HTTP page to HTTPS? especially those with basic forms (phone, email, name)??
I recently went to a conference where a speaker strongly urged us to migrate to HTTPS before January 2017. I don't see any other sites referencing to make the switch before January 2017. whats the deal? 😉
On-Page Optimization | | millenniumsi0 -
Why would changing 404 pages increase traffic by 9%?
Neil Patel claimed in this article that by creating a custom 404 page that links out to 25 to 50 random internal pages on the website, he was able to increase the traffic of Techcrunch by 9%. I'm a bit skeptical about this claim. A couple of questions: Is this theory sound? If you've personally tried this or have read other articles supporting Neil, I'd love to learn more. Would a big site like Techcrunch really have problems with Google not indexing all of its pages? Also, does getting more pages crawled help you get more traffic? Specifically, would it help a site like mine? For reference, my site gets an average of 12,040 pages crawled per day in last 90 days. Currently 28,922 pages have been indexed. Are there any possible downsides to trying this? Thanks!
On-Page Optimization | | Brand_Psychic0 -
404 errors on page urls that don't even exist
The Seomoz crawler found 404error of pages dont even exist. Ho can that be possible?? Pages like: URL: http://www.yoxo.it/catalog/seo_sitemap/category/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/falli/
On-Page Optimization | | yoxo0 -
Corporate Re-Branding, How to best lead into new website?
I have a company that is going to be moving away from a name they have had for 40 years, into a name they have had for the past 10. They are becoming a bigger brand in Canada and want to move away from the west coast name. http://www.wcmachinery.net/ is becoming obsolete. http://shearforce.ca/ will be taking over for both sites. Both of the websites are identically built by a previous web company. The current root domain metrics are shown in the image provided. What do you Mozzers feel would be the best way to phase out the old site without losing the little bit of domain authority wcmachinery.net has gained? west coast machinery is also well know to its customers, so getting the branded search traffic to the shearforce site is also a priority. I am fairly certain this can be handled easily in the about page where it explains the relationship between the two companies. As a side note I have not performed any SEO work on either site as of yet. So please do not judge me based on this site! Thanks for your insights, Randy edit edit
On-Page Optimization | | PixelgemsCreative0 -
Best practice for introducing new landing page to my site?
I have a client, and want to know the best way to add new, keyword specific landing pages to their site and link to it in a logical way that isn't spammy. Example: My homepage targets “Adelaide Cars” I also want to target “Melbourne Cars” which I would do via a targeted landing page. How then would I logically link to this landing page? As Google gets better at spotting un-natural content, I’d like to know how to introduce this new page to get the best traction. If I was to just create the page, it would not make sense to have it in the main navigation. Same goes from various industry type terms. Eg. pest control and exterminator. How do you target both and still have a logical sitemap and page structure that Google will like and make sense to users.
On-Page Optimization | | letgo3450 -
Is it ok to point internal links to index.html home page rather than full www
I thought I saw this somewhere on SEOmoz before but I was so busy by the time I got around to work on my SEO on my site, I realized I have this happening and can't recall if it is a problem which takes away from my ranking. If my www.website.com is ranking well but I have internal menu links pointing to www.website.com/index.html instead of www.website.com will that take away from my www.website.com rankings? Should I change all my menu links that point to /index.html to the full website url path www.website.com ?
On-Page Optimization | | Twinbytes0