Issue with Site Map - how critical would you rank this in terms of needing a fix?
-
A problem has been introduced onto our sitemap whereby previously excluded URLs are no longer being correctly excluded. These are returning a HTTP 400 Bad Request server response, although do correctly redirect to users.
We have around 2300 pages of content, and around 600-800 of these previously excluded URLs,
An example would be http://www.naturalworldsafaris.com/destinations/africa-and-the-indian-ocean/botswana/suggested-holidays/botswana-classic-camping-safari/Dates and prices.aspx (the page does correctly redirect to users).
The site is currently being rebuilt and only has a life span of a few months. The cost our current developers have given us for resolving this is quite high with this in mind. I was just wondering:
-
How much of a critical issue would you view this?
-
Would it be sufficient (bearing in mind this is an interim measure) to change these pages so that they had a canonical or a redirect - they would however remain on the sitemap.
Thanks
Kate -
-
Agree with Martijn. Technically, Google does not want any pages in the sitemap that 4xx or 3xx or have any other result than a 200.
I would say this, if your site is being rebuilt, having a sitemap that is accurate and that updates when you update the site is a basic requirement. The fact that there is a high cost for fixing this issue is baloney. It sounds like the devs did not build the site correctly the first time if they do not have a way to update the sitemap automatically.
You could generate the sitemap yourself
You can use tools like
http://tools.seochat.com/tools/online-crawl-google-sitemap-generator/
Or read tutorials on how to use Screaming Frog to create a sitemap
http://www.hmtweb.com/marketing-blog/dirty-sitemaps-how-to-download-crawl/
Frankly, the annual cost of Screaming Frog (about $150 a year) gets you so much more than just sitemaps. Buy Screaming Frog, have it generate your sitemap and ask the devs to upload it. If you have a site with several thousand pages, just running Screaming Frog monthly would help you find issues on your site that is well worth the cost. Search "Screaming Frog" here in the forums and you can see that this is one of the "swiss army knives" of technical SEO.
-
- It's not a perfect solution and if you can make the costs than I would always do it, but honestly I can't say this is very critical right now to fix. Google and other search engines want you to have a very high quality sitemap so that all pages in there will exist and work for them as well as for users but if a certain percentage of them won't work then it won't get you massively in trouble I'd say.
- I'm not really sure about this option as it doesn't sound like an actual fix at all so for now I would say don't do it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ranking issue for new website
Hi all, I have got a specific SEO challenge. 6 months ago, we started to build an eCommerce site (located in the UK). In order to speed up the site launch, we copied the entire site over from an existing site based in Ireland. Now, the new UK site has been running for 5 months. Google has indexed many pages, which is good, but we can't rank high (position: between 20-30 for most pages). We thought it was because of content duplication in spite of different regions. So we tried to optimize the pages for the UK site to make them more UK-related and avoid content duplication. I've also used schema to tell google it's a UK-based site and set up Google my business and got more local citations. Besides, If you could give me any suggestions, it'd be perfect.
Intermediate & Advanced SEO | | Insightful_Media
Thank you so much for your time and advice.1 -
How to rank for a keyword
Hi guys, I'm looking to rank a new blog for a search term which we currently already rank #1 for. I want to create a blog which provides a better solution to a searchers query and knock our current #1 page for this new one. Is there a way to do this simply without losing the real-estate the currently ranked #1 page has already accumulated? Or is just a matter of working on this new blog to find it's own way to the top? Thanks in advance, James
Intermediate & Advanced SEO | | jameseacott0 -
Can't understand our Ranking; DA, PA and On Page all better than Competiton; Ranking no where
Our rankings are up and down but our domain is clean, DA and PA good
Intermediate & Advanced SEO | | AlexSTUDIO18
and there is really in depth content which is all original. We are at a bit of a loss; The site is
http://www.fightstorepro.com I can use the phrase "Boxing Gloves" as an example
http://www.fightstorepro.com/gear/gloves/boxing-gloves.html PA 26 , DA29
Good original content; Video content, On page grade A
Not ranking in top 50 places?? The competitor in Pos4 is not matching our placing Anyone shed any light on this?0 -
How do I get my site to rank in the right country?
I work with an organization in the United States that purchased a 3 letter .com domain from another company based in the Netherlands. After transferring the domain and setting up the new site we've completed the following: Registered the site in Webmaster Tools and selected the United States as the international target target Used Schema Markup on the US business address Built links from US based websites Setup a Google My Business site and verified the address All site content is in english Requested removal of links from sites in the Netherlands that were linking to the old website Disavowed all links from the Netherlands that refused to remove links to the old website Despite these actions, our website rarely shows up in Google.com search results. However, we're in the top 10 for many of the same queries when performing searches at Google.nl - Interestingly enough, if we set our location for the US and do a search in Google.nl we do not show up in the search results.
Intermediate & Advanced SEO | | brianspatterson
It has been about 18 months since we've completed these actions and Google still assumes our website is best served and targeted towards people in the Netherlands. Is there anything else we can do to fix this?Thanks!0 -
Manual action penalty revoked, rankings still low, if we create a new site can we use the old content?
Scenario:
Intermediate & Advanced SEO | | peteboyd
A website that we manage was hit with a manual action penalty for unnatural incoming links (site-wide). The penalty was revoked in early March and we're still not seeing any of our main keywords rank high in Google (we are found on page 10 and beyond). Our traffic metrics from March 2014 (after the penalty was revoked) - July 2014 compared to November 2013 - March 2014 was very similar. Question: Since the website was hit with a manual action penalty for unnatural links, is the content affected as well? If we were to take the current website and move it to a new domain name (without 301 redirecting the old pages), would Google see it as a brand new website? We think it would be best to use brand new content but the financial costs associated are a large factor in the decision. It would be preferred to reuse the old content but has it already been tarnished?0 -
Need some expert help – My Client bought out competitor and now wants to completely duplicate the current site with the same stock & categories using the Competitor brand
I am the SEO consultant for a large online homewares store. This company currently ranks very well in Google. I can PM the domain name if anyone needs however i don't want to post it on this forum. The company has bought out a competitor and plan to use the same warehouse, same products, and same back-end system as the current site, so they want to completely duplicate the current website. Titles, meta descriptions, product descriptions will all be renamed/rewritten/reworded (however keep in mind there are not many ways to reword a 3 piece saucepan set) Pricing will mostly be the same (some difference though), images cannot be renamed, categories cannot be renamed... the structure of the site will be exactly the same... placement etc. (however will have different banners, logo etc.) I personally don't believe the new site will rank, because it will be too similar. Can someone please offer me a 2nd opinion... Thanks
Intermediate & Advanced SEO | | ryanlenton0 -
New site started ranking, lost ground after 301...?
Hi everyone. So we decided to re-brand a website starting with a fresh domain and fresh website (new everything). The first 4 weeks the site was performing really well and started bringing organic traffic for several targeted keywords (small amounts, but something). Around 5 weeks later, we decided to perform a 301 Redirect from an older site we had that really hadn't changed since 2007. As soon as we performed the 301, we started loosing ground on the keywords that were starting to perform. I may help to mention that we were targeting different keywords on the new site, versus the keywords/industry that were targeted on the older site... because we were focusing on another but similar industry. Now, 10 weeks later we are still not showing up for the keywords that we were starting to make headway on in the first four weeks of starting.... any ideas why? suggestions? The 301's were performed properly. We verified them, and we check Google WMT for any additional ones that may still be out there, but try to fix them as soon as possible. We have grown the site from just a few pages to over 60+ pages in the last 2 months with unique and fresh content targetting those keywords... Thank you in advance
Intermediate & Advanced SEO | | co.mc0 -
Large Site SEO - Dev Issue Forcing URL Change - 301, 302, Block, What To Do?
Hola, Thanks in advance for reading and trying to help me out. A client of mine recently created a large scale company directory (500k+ pages) in Drupal v6 while the "marketing" type pages of their site was still in manual hard-coded HTML. They redesigned their "marketing" pages, but used Drual v7. They're now experiencing server conflicts with both instances of Drupal not allowing them to communicate/be on the same server. Eventually the directory will be upgraded to Drupal v7, but could take weeks to months the client does not want to wait for the re-launch. The client wants to push the new marketing site live, but also does not want to ruin the overall SEO value of the directory and have a few options, but I'm looking to help guide them down the path of least resistance: Option 1: Move the company directory onto a subdomain and the "marketing site" on the www. subdomain. Client gets to push their redesign live, but large scale 301s to the directory cause major issues in terms of shaking up the structure of the site causing ripple effects into getting pulled out of the index for days to weeks. Rankings and traffic drop, subdomain authority gets lost and the company directory health looks bad for weeks to months. However, 301 maintains partial SEO value and some long tail traffic still exists. Once the directory gets moved to Drupal v7, the directory will then cancel the 301 to the subdomain and revert back to original www. subdomain URLs Option 2: Block the company directory from search engines with robots.txt and meta instructions, essentially cutting off the floodgates from the established marketing pages. No major scaling 301 ripple effect, directory takes a few weeks to filter out of the index, traffic is completely lost, however once drupal v7 gets upgraded and the directory is then re-opened, directory will then slowly gain back SEO value to get close to old rankings, traffic, etc. Option 3: 302 redirect? Lose all accumulate SEO value temporarily... hmm Option 4: Something else? As you can see, this is not an ideal situation. However, a decision has to be made and I'm looking to chose the lesser of evils. Any help is greatly appreciated. Thanks again -Chris
Intermediate & Advanced SEO | | Bacon0