Issue with Site Map - how critical would you rank this in terms of needing a fix?
-
A problem has been introduced onto our sitemap whereby previously excluded URLs are no longer being correctly excluded. These are returning a HTTP 400 Bad Request server response, although do correctly redirect to users.
We have around 2300 pages of content, and around 600-800 of these previously excluded URLs,
An example would be http://www.naturalworldsafaris.com/destinations/africa-and-the-indian-ocean/botswana/suggested-holidays/botswana-classic-camping-safari/Dates and prices.aspx (the page does correctly redirect to users).
The site is currently being rebuilt and only has a life span of a few months. The cost our current developers have given us for resolving this is quite high with this in mind. I was just wondering:
-
How much of a critical issue would you view this?
-
Would it be sufficient (bearing in mind this is an interim measure) to change these pages so that they had a canonical or a redirect - they would however remain on the sitemap.
Thanks
Kate -
-
Agree with Martijn. Technically, Google does not want any pages in the sitemap that 4xx or 3xx or have any other result than a 200.
I would say this, if your site is being rebuilt, having a sitemap that is accurate and that updates when you update the site is a basic requirement. The fact that there is a high cost for fixing this issue is baloney. It sounds like the devs did not build the site correctly the first time if they do not have a way to update the sitemap automatically.
You could generate the sitemap yourself
You can use tools like
http://tools.seochat.com/tools/online-crawl-google-sitemap-generator/
Or read tutorials on how to use Screaming Frog to create a sitemap
http://www.hmtweb.com/marketing-blog/dirty-sitemaps-how-to-download-crawl/
Frankly, the annual cost of Screaming Frog (about $150 a year) gets you so much more than just sitemaps. Buy Screaming Frog, have it generate your sitemap and ask the devs to upload it. If you have a site with several thousand pages, just running Screaming Frog monthly would help you find issues on your site that is well worth the cost. Search "Screaming Frog" here in the forums and you can see that this is one of the "swiss army knives" of technical SEO.
-
- It's not a perfect solution and if you can make the costs than I would always do it, but honestly I can't say this is very critical right now to fix. Google and other search engines want you to have a very high quality sitemap so that all pages in there will exist and work for them as well as for users but if a certain percentage of them won't work then it won't get you massively in trouble I'd say.
- I'm not really sure about this option as it doesn't sound like an actual fix at all so for now I would say don't do it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SERP Ranking
Hi everyone, I'm struggling to figure out why our website has dramatically changed and is ranking far worse than ever. We were top for around 10 keywords, and had over 4 high traffic keywords in position 1. I think the website needs updating (as it's magento 1.7), and that the competition is increasing. Taking our eye off the ball has been a huge mistake. Could anyone advise on what the next steps would be to start increasing to a better serp rank? These two are own main ranking pages:
Intermediate & Advanced SEO | | mg33dev
https://www.mediagenic.co.uk/
https://www.mediagenic.co.uk/budget-roller-banner.html Any advice or pointers would be absolutely awesome. Thanks0 -
Why does some sites rank with no seo
Why is it that some site rank with zero efforts? I have been working on some seo for a while on my main site and i have been getting more info competition analysis with sem and moz. Looking at the states from this website which tends to popup often in the searches on page 1-2 before my site. This site is not keyword optimized, meaning they arent even trying to rank.
Intermediate & Advanced SEO | | CooperStrzelecki
There is no content, articles etc.,
6 backlinks (nothing powerful just 2 directory links and 2 from developer)
Site really isnt even designed to get traffic as its a trade only ecommerce website
I doubt they are hiding anything as far as backlinks etc. as it will get them too many visitors they dont want
The city i am searching isnt even on the page (it is a city within a city so maybe google still relates it)
PA 24 DA 15 Now my site:
Optimized reasearched keywords
175 backlinks
All my main pages have content with images, alt tags, internal linking
full of content, blogs, videos, products (probably 4000, could a site being too big be an issue?)
Site gets regular updates
I probably have 200 citations
All the social media which gets done often
PA 32 DA 20 They do get a good bit of traffic but that is probably the only thing i would see but it would be direct traffic mostly i believe as it would be people going to order regularly since it is a print reseller. They may have some age on me 15 vs 8 years. Could it be some kind of penalty i am not sure about lingering? According to what i know to check everyything looks ok, no shady links accoding to sem. I am working more and more on all the pages but this competittion site really doesnt have crap going on probably 8 pages and 1 page does all the ordering. What the hell does google want from me exactly!0 -
Site re-design, full site domain A/B test, will we drop in rankings while leaking traffic
We are re-launching a client site that does very well in Google. The new site is on a www2 domain which we are going to send a controlled amount of traffic to, 10%, 25%, 50%, 75% to 100% over a 5 week period. This will lead to a reduction in traffic to the original domain. As I don't want to launch a competing domain the www2 site will not be indexed until 100% is reached. If Google sees the traffic numbers reducing over this period will we drop? This is the only part I am unsure of as the urls and site structure are the same apart from some new lower level pages which we will introduce in a controlled manner later? Any thoughts or experience of this type of re-launch would be much appreciated. Thanks Pete
Intermediate & Advanced SEO | | leshonk0 -
Can I add Title/Description tags to site map
I have started working on a website that it written in JAVA. It has 26 URL's But because of the way it is written it is all shown on the home page code and does not have the ability to add unique title and description tags. Is there a work around for SEO on websites like this aside from adding content? I was wondering if there is a way to submit a sitemat with title and description tags. Any advice? Chris.K
Intermediate & Advanced SEO | | CKerr0 -
Homepage not ranking in Google AU, but ranking in Google UK?
Hey everyone, My homepage has not been ranking for it's primary keyword in Google Australia for many months now. Yesterday when I was using a UK Proxy and searching via Google UK I found my homepage/primary keyword ranked on page 8 in the UK. Now in Australia my website ranks on page 6 but it's for other pages on my website (and it always changes from different page to page). Previously my page was popping up at the bottom of page 1 and page 2. I've been trying many things and waiting weeks to see if it had any impact for over 4 months but I'm pretty lost for ideas now. Especially after what I saw yesterday in Google UK. I'd be very grateful if someone has had the same experience of suggestions and what I should try doing. I did a small audit on my page and because the site is focused on one product and features the primary keyword I took steps to try and fix the issue. I did the following: I noticed the developer had added H1 tags to many places on the homepage so I removed them all to make sure I wasn't getting an over optimization penalty. Cleaned up some of my links because I was not sure if this was the issue (I've never had a warning within Google webmaster tools) Changed the title tags/h tags on secondary pages not to feature the primary keyword as much Made some pages 'noindex' to try and see if this would take away the emphases on the secondary pages Resubmitted by XML sitemaps to Google Just recently claimed a local listings place in Google (still need to verify) and fixed up citations of my address/phone numbers etc (However it's not a local business - sells Australia wide) Added some new backlinks from AU sites (only a handful though) The only other option I can think of is to replace the name of the product on secondary pages to a different appreciation to make sure that the keyword isn't featured there. Some other notes on the site: When site do a 'site:url' search my homepage comes up at the top The site sometimes ranked for a secondary keyword on the front page in specific locations in Australia (but goes to a localised City page). I've noindexed these as a test to see if something with localisation is messing it around. I do have links from AU but I do have links from .com and wherever else. Any tips, advice, would be fantastic. Thanks
Intermediate & Advanced SEO | | AdaptDigital0 -
3 Wordpress sites 1 Tumblr site coming under 1domain(4subdomains) WPMU: Proper Redirect?
Hey Guys, witnessSF.org (WP), witnessLA.org(Tumblr), witnessTO.com(WP), witnessHK.com(WP), and witnessSEOUL.com(new site no redirects needed) are being moved over to sf.ourwitness.com, la.ourwitness.com and so forth. All under on large Wordpress MU instance. Some have hundreds of articles/links others a bit less. What is the best method to take, I understand there are easy redirects, and the complete fully manual one link at a time approach. Even the WP to WP the permalinks are changing from domain.com/date/post-name to domain.com/post-name? Here are some options: Just redirect all previous witinessla.org/* to la.ourwitness.org/ (automatic direct all pages to home page deal) (easiest not the best)2) Download Google Analytics top redirected domains about 50 urls have significant ranking and traffic (in LA's sample) and just redirect those to custom links. (most bang for the buck for the articles that rank manually set up to the correct place) 3) Best of the both worlds may be possible? Automated perhaps?I prefer working with .htaccess vs a redirect plugin for speed issues. Please advise. Thanks guys!
Intermediate & Advanced SEO | | vmialik0 -
Is it Wortwhile to have a HTML site map for a Large Site
We are a large, enterprise site with many pages (some on our CMS and some old pages that exist outside our CMS). Every month we submit various an XML site map. Some pages on our site can no longer be found via following links from one page to another (orphan pages). Some of those pages are important and some not. Is it worth our while to create a HTML site map? Does any one have any recent stats or blog posts to share, showing how a HTML site map may have benefited a large site. Many thanks
Intermediate & Advanced SEO | | CeeC-Blogger0 -
If I had an issue with a friendly URL module and I lost all my rankings. Will they return now that issue is resolved next time I'm crawled by google?
I have 'magic seo urls' installed on my zencart site. Except for some reason no one can explain why or how the files were disabled. So my static links went back to dynamic (index.php?**********) etc. The issue was resolved with the module except in that time google must have crawled my site and I lost all my rankings. I'm nowher to be found in the top 50. Did this really cause such an extravagant SEO issue as my web developers told me? Can I expect my rankings to return next time my site is crawled by google?
Intermediate & Advanced SEO | | Pete790