Issue with Site Map - how critical would you rank this in terms of needing a fix?
-
A problem has been introduced onto our sitemap whereby previously excluded URLs are no longer being correctly excluded. These are returning a HTTP 400 Bad Request server response, although do correctly redirect to users.
We have around 2300 pages of content, and around 600-800 of these previously excluded URLs,
An example would be http://www.naturalworldsafaris.com/destinations/africa-and-the-indian-ocean/botswana/suggested-holidays/botswana-classic-camping-safari/Dates and prices.aspx (the page does correctly redirect to users).
The site is currently being rebuilt and only has a life span of a few months. The cost our current developers have given us for resolving this is quite high with this in mind. I was just wondering:
-
How much of a critical issue would you view this?
-
Would it be sufficient (bearing in mind this is an interim measure) to change these pages so that they had a canonical or a redirect - they would however remain on the sitemap.
Thanks
Kate -
-
Agree with Martijn. Technically, Google does not want any pages in the sitemap that 4xx or 3xx or have any other result than a 200.
I would say this, if your site is being rebuilt, having a sitemap that is accurate and that updates when you update the site is a basic requirement. The fact that there is a high cost for fixing this issue is baloney. It sounds like the devs did not build the site correctly the first time if they do not have a way to update the sitemap automatically.
You could generate the sitemap yourself
You can use tools like
http://tools.seochat.com/tools/online-crawl-google-sitemap-generator/
Or read tutorials on how to use Screaming Frog to create a sitemap
http://www.hmtweb.com/marketing-blog/dirty-sitemaps-how-to-download-crawl/
Frankly, the annual cost of Screaming Frog (about $150 a year) gets you so much more than just sitemaps. Buy Screaming Frog, have it generate your sitemap and ask the devs to upload it. If you have a site with several thousand pages, just running Screaming Frog monthly would help you find issues on your site that is well worth the cost. Search "Screaming Frog" here in the forums and you can see that this is one of the "swiss army knives" of technical SEO.
-
- It's not a perfect solution and if you can make the costs than I would always do it, but honestly I can't say this is very critical right now to fix. Google and other search engines want you to have a very high quality sitemap so that all pages in there will exist and work for them as well as for users but if a certain percentage of them won't work then it won't get you massively in trouble I'd say.
- I'm not really sure about this option as it doesn't sound like an actual fix at all so for now I would say don't do it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How To Rank Our Featured Snippet - What Changes Are Needed On Our Page?
I've read a number of articles that have been helpful, but most of them are specifically still just trying to prove the value of the snippets and more recently show you how to find what search terms to rank for. What I'm really struggling with is exactly 'How do we rank for them, when we already have the #1 position and the featured snippet is going to another site'? Let me break this down a bit more: 1. We are measuring the 'SERP Features' within Moz Pro Tools and I've identified ~300 pages where there is a 'Featured Snippet' but I don't have the feature. 2. In a good portion of these, I'm outranking the site that has the 'Featured Snippet'. So I can compare my site, side by side to the 'Featured Snippet'. Now that I have the question, my ranking and the competition all in front of me. What changes are recommended I implement on our page? Is there a recommended process to follow?
Intermediate & Advanced SEO | | fabfrug0 -
Google is alternating what link it likes to rank on wordpress site and
Hi there, I'm experiencing a problem where google is pick and choosing different links structures to rank my Wordpress site for my main keywords. The site had pretty good #1 rankings for a long time but recently I noticed Google is choosing to rank the page in one of two ways. Let me just say that the original way where it held good rankings looked like this for example: flowers.com/the-most-beautiful-wedding-bouquets/ this is just an example it' is not my site. And when google decides to switch it up it uses this link structure:flowers.com > weddings (this still points to this link flowers.com/the-most-beautiful-wedding-bouquets when I hover my mouse over it) however this link structure that never appeared before and now does, usually has much lower rankings. Please note it's not both link structures being ranked at the same time for the keywords. It's one or the other that google is currently alternating in ranking and I believe it's hurting the sites position.
Intermediate & Advanced SEO | | z8YX9F80
I'm not sure if this is a wordpress settings thats gone wrong or what the problem is but I do know when shows the expanded and descriptive link structure flowers.com/the-most-beautiful-wedding-bouquets the rankings are higher and in 2nd place. I'm hoping by rectifying this I can regain back my position. I'm very grateful for any insight you could offer on why this is happening and how I could fix it. Thank you. PS Wordpress site has several SEO plugins0 -
I am launching a new site, what I need to know about backlinks
Hi 🙂 soon I am launching a new site. It will be focused on topics: Android, smartphones, reviews etc... First I will buy a good looking WP theme, it will be responsive theme. after that when I set up all in theme, after that I will start writing. I will try to write interesting and good content.... This content I will share on my pages: Facebook, Twitter, Google+ and Youtube. So only remains, how to get good backlinks ? good backlinks for Google ? Blog commenting is not good, right ? Writing on relative forums and posting links as source of that information ? Contacting other relative sites for sharing my content ? Am I missed something or not ? In your opinion, which is the best way of getting good backlinks ? And what is the best solution in the beginning, because no one will know my site because my site is new. Thank you 🙂
Intermediate & Advanced SEO | | Ivek990 -
New site now links disappearing in Open Site Explorer and GWT
We launched a new site at the beginning of December 2012 and carefully 301'd all URLs from the old site to the new (custom CMS on old site wordpress on new). Our rankings have slipped quite badly but the most worrying thing is that we used to have about 1200 backlinks according to GWT/OSE before the new site launched and now we're down to about 30. Can anyone help shed some light on this please? The site is www.littleoneslondon.co.uk A few things that might help: 1. We were getting a lot of links through our job feeds (it's a nanny recruitment site) on indeed and trovitt, for some reason no new ones from these have appeared in site explorer and all the old jobs are gone completely. 2. We had 1000s of not found errors in google webmaster tools and once these were redirected and marked as fixed this is when the links disappeared. 3. We are getting quite a few 504 errors on the site due to an old proxy redirect (/blog was hosted on a different server on the old site and has not been removed yet), this will be fixed tomorrow but could this be a factor? 4. The developer seems to have redirected all the links through wordpress directly some how (I don't see any redirect plugins but there are lots of pages called 'redirect'). There are no references in the htaccess file for any redirects other than from the /blog folder that the wordpress instance sits in. Sorry for the long post, I hope I've given any details you'd need and I really appreciate any help anyone can give. Thanks, Karl
Intermediate & Advanced SEO | | Bdig0 -
Does Mobile optimised site improve ranking and how to index it faster?
Hi i have several question with regards to mobile optimised site: Does having a mobile optimised site improve ranking in SERP? How can we push/index mobile optimised sites to users searching on mobile sites faster? e.g. returning m.abc.com or abc.com/m to users seraching on mobile earlier.
Intermediate & Advanced SEO | | FWSBIO0 -
Can a Hosting provider that also hosts adult content sites negatively affect our SEO rankings on a non-adult site hosted on same platform?
We're considering moving a site to a host that also offers hosting for adult websites. Can this have a negative affect on SEO, if our hosting company is in any way associated with adult websites?
Intermediate & Advanced SEO | | grapevinemktg0 -
My site links have gone from a mega site links to several small links under my SERP results in Google. Any ideas why?
A site I have currently had the mega site links on the SERP results. Recently they have updated the mega links to the smaller 4 inline links under my SERP result. Any idea what happened or how do I correct this?
Intermediate & Advanced SEO | | POSSIBLE0 -
Need help/insights. Site SEO = OK, Authority BLs = OK, Rank page #1\. How to reach pos #1?
Hi there! Some doubts are confusing my head and need some assistence from you to get on the right track. I'll explain my situation and want to hear from you what do you really recommend for med/long term permanent results. 1 - I have a PR2 (.com.br) domain; 2 - I'm talking about little/med competition micro-niche keywords; 3 - I got all pages I want to, indexed (I have a well SEO constructed website with internal link building); 4 - If a keyword has average competition, I'll already start ranking in page #3 on the SERP's; For a few low competition keywords I start on page #1; 5 - I do a little whitehat link building, 1 or 2 backlinks on authority sites and then like 15 days later I came to page #1, generally on position 9/10; And then I got stucked 🙂 No more authority sites where I can get backlinks... I do some posts on the company twitter/facebook page's, but they are no follow, so I don't really now if this can help. (never see a SERP result). I did some "blackhat" stuff to see if it really work: I can say for sure the "profile backlinks" that we can buy from some sites doesn't work (maybe it's just for me). I can't see it on webmaster tool and neither my ranks changed since I bought a pack of 100 links (the links are working, I see it one by one) to test. Maybe the problem is about the domains, cause my site is .com.br and I'm buying .com profile links. I guess google understand backlinks from .com.br more valuable for my sites. Back to whitehat: I wrote some articles and posted it the right way, of course on .com.br articles sites, got it indexed and can see the backlink on webmaster tool, but no change on SERP's. (maybe this can be a long term result and I'm not seeing it yet). I'm really "scratching my hand" to do some blackhat stuff, but I don't want to lose what I already have done... I heard a lot about scrapebox but doesn't fell confortable to spam as hell a lot of blogs. I really want long term permanent results (my sites are totally whitehat/corporate sites). Can you expert guys give me some point to where I need to "walk" now to improve the SERP's? I never reached top #1 and want to try to rank at least one time to understand how this can be made... I'm thinking now to pay someone to rewrite 20 copies of an article and up it on some sites, to see if 20 can improve something. But still no confident, because it will cost like $100 for a good writer do it for me on my language. Maybe I can do better things with 100 bucks. I guess I did the path right: Internal SEO -> got indexed -> backlinking from authorities -> new articles backlinks to me (is it ok at this position or no?) -> (what next ?) I know SEO is a hard/never ending work, but what I'm trying to get cleaned on my head is the path of the work (if a right path really exists). Every word will be apreciated. What do you can suggest to me to try now? (please give me a hint to see SERP's results 🙂 if I feel that something worked, no matter how it can cost to me, but I'll pay for the work happily) Sorry if I'm a little confusing, english isnt' my first language. Thanks.
Intermediate & Advanced SEO | | azaiats20