Historic issue with incomplete indexing
-
Hi there
We run quite a big site in the UK in the commercial real-estate space.
Historically we have always had a challenge getting our "primary" landing pages indexed, which are location based property result pages.
e.g. https://realla.co/to-rent/commercial-property/oxford
For example, for the "towns" category we have 8,549 submitted in our xml sitemap, with only 3,171 indexed. This is a general issue across all our sitemaps. 120k submitted, 80k indexed. Our pages are linked through breadcrumbs, and nearby links.
In the new search console these pages are reported as "crawled - currently not indexed"
These all sit under the folder:
site:https://realla.co/to-rent/commercial-property/*
site:https://realla.co/to-rent/office/*
We have done extensive work to optimise performance, including AMP pages.
Each location page has many details pages for individual properties e.g.
https://realla.co/to-rent/details/0ffbbd0a1a1147edb8847c5ce6179509
One action we have remaining is to nest the details under the locations pages, which may help. These details pages are indexed fully.
Any feedback much appreciated
-
Hi Ian,
The details URL should ideally have keywords in it, getting property name in details page URL would be of great help, like : https://realla.co/to-rent/details/Office-to-let-John-Eccles-House-Robert Robinson-Avenue-Oxford-Science-Park-Oxford-OX4-4GP
About the category (locations in your case), you are submitting too many of them, your URL structure needs to re-structured, there is work to be done there and sitemap updated according to that. For example:
https://realla.co/to-rent/commercial-property/
can be changed to
https://realla.co/commercial-property-to-rent/
I hope this helps, let me know if you have further queries.
Regards,
Vijay
-
Thanks for your reply
We are just about to nest the "details" pages under the results path e.g. /to-rent/commercial-property/newbury/details/1294321739712973129 etc so it sits under the right location.
I think this is in line with your recommendation.
We have alot of individual sitemap files, should these be consolidated?
-
Hi Ian,
I have analyzed the website in detail, the problem seems to be that you are not giving any differentiation to search engine bots between important category/sub-category(in your case different locations) pages compared to product pages (in your case property details page). The location pages URL structure and their sitemap submission strategy can be re-worked to get the desired results.
Another scope of improvement is in URL structure for property details page **For example, **
https://realla.co/to-rent/details/0ffbbd0a1a1147edb8847c5ce6179509 should be https://realla.co/to-rent/details/Office-to-let-John-Eccles-House-Robert Robinson-Avenue-Oxford-Science-Park-Oxford-OX4-4GP
Your site structure is huge, and it must be getting dynamic links generated or removed, you need to be careful with the site structure and how often to submit sitemap.
I hope this helps. Let me know if you have further queries, I will be happy to help.
Regards,
Vijay
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
403 Errors Issue
Hi, all! I've been working with a Wordpress site that I inherited that gets little to no organic traffic, despite being content rich, optimized, etc. I know there's something wrong on the backend but can't find a satisfactory culprit. When I emulate googlebot, most pages give me a 403 error. Also, google will not index many urls which makes sense and is a massive headache. All advice appreciated! The site is https://www.diamondit.pro/ It is specific to WP Engine, using GES (Global Edge Security) and WPWAF
Technical SEO | | SimpleSearch0 -
Woocommerce URL Structure Issue
Hi everyone ! To put you in context, I am doing an audit on an E-Commerce site selling auto parts with WooCommerce. I have some concerns regarding the url structure and here's why: Product category page url: /auto/drivetrain/cv-axle-shaft-assembly/
Technical SEO | | alexrbrg
Product page url included in the product category page: /product/acura-integra-cv-axle-shaft-90-01-honda-civic/ The way I see my situation is that the product page is considered by Google as an intern link and not as a page included in in the subfolder of the category page. 1. Am I right?
2. If yes, is there a solution to fix the issue with woocommerce to improve the category page ranking ? Thanks y'all !0 -
An odd duplicate content issue...
Hi all, my developers have just assured me that nothing has changed form last week but in the today's crawl I see all the website duplicated: and the difference on the url is the '/' so basically the duplicated urls are: htts://blabla.bla/crop htts://blabla.bla/crop/ Any help in understanding why is much appreciated. thanks
Technical SEO | | LeadGenerator0 -
Does server affect indexing speeds?
A bit of a strange question this one: I have a domain which, when on my Dutch server, can get new blog posts indexed and ranking in less than 10 mins using the pubsubshubbub plugin. However, I moved the blog and domain to a UK dedicated server and continued to post. Days later none of these posts were indexed. I then moved the domain back to the Dutch server to test this, I posted in the blog and once again, indexed and ranking in 20 mins or so. To cut a long and tedious story short; In a bid to be closer to my customers I moved the domain to a UK VPS three days back. I posted but no posts are indexed. Anyone else experienced anything like this? Generally I don't move domains back and forward so much but wanted to test this out. The Ducth server is a 16 core 24gb Direct Admin dedicated, the two UK servers were both running Cpanel. I understand that it would be best to host as close to possible to the customers but the hardship of getting posts indexed in the UK is becoming a problem. Thanks, Carl
Technical SEO | | Grumpy_Carl1 -
Database Driven Websites: Crawling and Indexing Issues
Hi all - I'm working on an SEO project, dealing with my first database-driven website that is built on a custom CMS. Almost all of the pages are created by the admin user in the CMS, pulling info from a database. What are the best practices here regarding SEO? I know that overall static is good, and as much static as possible is best, but how does Google treat a site like this? For instance, lets say the user creates a new page in the CMS, and then posts it live. The page is rendered and navigable, after putting together the user-inputed info (the content on the page) and the info pulled from the database (like info pulled out to create the Title tag and H1 tags, etc). Is this page now going to be crawled successfully and indexed as a static page in Google's eyes, and thus ok to start working on rank for, etc? Any help is appreciated - thanks!
Technical SEO | | Bandicoot0 -
I have a site that has both http:// and https:// versions indexed, e.g. https://www.homepage.com/ and http://www.homepage.com/. How do I de-index the https// versions without losing the link juice that is going to the https://homepage.com/ pages?
I can't 301 https// to http:// since there are some form pages that need to be https:// The site has 20,000 + pages so individually 301ing each page would be a nightmare. Any suggestions would be greatly appreciated.
Technical SEO | | fthead90 -
Https indexed - though a no index no follow tag has been added
Hi, The https-pages of our booking section are being indexed by Google. We added But the pages are still being indexed. What can I do to exclude these URL's from the Google index? Thank you very much in advance! Kind regards, Dennis Overbeek ACSI Publishing | dennis@acsi.eu
Technical SEO | | SEO_ACSI0 -
Sitemaps - Format Issue
Hi, I have a little issue with a client site whose programmer seems kind of unwilling to change things that he has been doing a long time. So, he has had this dynamic site set up for a few years and active in google webmaster tools and others, but is not happy with the traffic it is getting. When I looked at webmaster tools I see that he has a sitemap registered, but it is /sitemap.php When I said that we should be offering the SE's /sitemap.xml his response is that sitemap.php checks the site every day and generates /sitemap.xml, but there is no /sitemap.xml registered in webmaster tools. My gut is telling me that he should just register /sitemap.xml in webmaster tools, but it is a hard sell 🙂 Anyone have any definitive experience of people doing this before and whether it is an issue? My feeling is that it doesn't need to be rocket science... Any input appreciated, Sha
Technical SEO | | ShaMenz0