Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
-
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep.
I just wanted to know the possible reasons and has anyone encountered this before?
-
Hi Moosa, do you have an update on this issue?
-
Things I check:
- header check (I've had pages appear 200 OK in webconfs and then massive errors in Browseo so I often check both.)
- Screaming Frog check - does the direct URL start a proper crawl?
- Make sure it's not blocked in WMT under removed pages (some website owners and/or bad SEOs try to demote sitelinks by removing the URL.)
- Meta tags & robots.txt both of which I assume you already checked.
- Duplicate content being indexed. Take a random sentence & throw it in quotes on Google. If a 100% dupe version comes up on their facebook page, archive.is, or some directory it may just need a content refresh to push through.
If you check all 5 of those off, then I'd definitely like to see the URL.
-
If that doesn't work, please share the page URL.
-
Thanks Drik,
I have double check the accessibility of page to crawlers and bots and pages does have decent amount of internal and external links pointing to the page.
Thanks for the Google fetch idea, i am going to try it to see if it works and update you accordingly.
-
Hi
I guess you have already checked that the page is accessible for crawlers and has sufficient internal links to be discovered by the bots (if not - try crawling the site with Screaming Frog)
If it's only one page that isn't picked up you could try a fetch like google & submit to index to force the page being indexed.
It could be that Google considers this page as less important and crawls with lower frequency compared to the other pages (to deep in the site, to few links, not enough content,...)
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Internal links are not indexed of the website
Some internal links are indexed and some not of the same page of the website, what is the so and what is the reason behind?
On-Page Optimization | | renukishor10 -
Why will a Page not rank or improve on a website?
My website is performing well, but I have a couple of popular Brand pages that no matter what I do will not rank organically, they are being indexed, but will just not perform or improve. The rest of the pages throughout the website all seem to be fine, can anyone advise if they have experienced a similar problem or advise why this would happen or even suggest what I should do to fix the problem, we have even gone as far to create a new page and delete the old one, and it still hasn't worked. I have used various tools to test the page and variations of search terms and although it is appearing in Google it is not ranked, although it is ranking and visible in Yahoo and Bing. Any help and advise would be appreciated. Jacqui
On-Page Optimization | | EurekaSolutions0 -
Regarding Google Title 'Width' and changing Meta Titles w/o Penalty?
A vast majority of pages on my site are now too wide (the character count was fine prior to the March update). I want to go through and update them so they display properly and are not too wide.However, I am concerned, as my understanding was that changing Meta Titles is dangerous and can have a negative effect on your rankings and can cause real issues. Is this an opportunity to change my Titles all-together without any kind of penalty? Or can I only trim the end part? In summary: 1. Can I edit all of my Meta Titles without affecting my rankings? 2. If no, how do I edit them properly to fit within the proper width and not cause any issues? 3. If yes, I can go through and change all my Meta Titles to whatever extent and optimize them to reflect latest best practices? There are changes I wanted to make to all my meta titles but I've been afraid to... due to fear of rankings drops etc Any help with this would be greatly appreciated
On-Page Optimization | | lawfirm0 -
Deleted pages still registering as 404 pages.
I have a an all html site that I can only work on through the ftp. The previous marketing company ran a script that built thousands of location landing pages, but all they did was change the tags and headers and the keywords in the pages, other than that they are all duplicate pages. I removed them, but Google is reading them as 404 pages. How do I tell Google those pages don't exist? or do I just need to let the bots crawl it a few times and it will see that eventually?
On-Page Optimization | | SwanJob0 -
What's the best way to handle crawling of photo gallery?
When you have a photo gallery with many search filters and loads and loads of pages, is it best to block all the filters and use google's pagination code? Ex: http://photo.net/gallery/photocritique/filter This site has pages for many different queries. While the page titles are unique, the pages are showing duplicated content.
On-Page Optimization | | cakelady0 -
Client has 2 websites... won't do away with old site.
Ok my situation is: We have built a new site for a client and they would like to keep both sites working. Should I just attempt to optimize both sites to rank well in their area? Or should I be suggesting a 301 redirect from the old domain to the new? Other options? Thanks in advance
On-Page Optimization | | IntermarkGroup0 -
How does a keyword get crawled and pointed at a certain page
I was wondering if you can give me some insight on how a keyword that I put on my campaign gets linked to a specific URL on my website by SEOMoz or Google. For example: updating a brick fireplace is my keyword. On the campaign when I am looking at my on page optimization, the URL assigned (or given) to it is my homepage. How is this determined and is there a way around it and or directing it to the correct page? Thanks
On-Page Optimization | | SammyT0