XML Sitemap Question!
-
Hi All,
I know that the sitemaps.xml URL must be findable but what about the sitemaps/pageinstructions.xml URL?
Can we safely noindex the sitemaps/pageinstructions.xml URL?
Thanks!
Yael
-
In regards to an "XML site map"- it's essential to have a site-map on your company website, that's because it makes it much easier for a Google Bot, to find the pages/ blog posts to crawl and index these if you have a XML site map.
We added a Yoast SEO site map to our Bristol Garden Room company website, in Bristol to ensure that all the blog posts / main pages were getting indexed.
-
Yes, you can noindex practically anything using the robots.txt file
-
Hey
Yes, you can safely do that. In the end, you mostly want to make sure that the right pages are being Crawled > Indexed. If that requires certain (XML) files to be no-indexed, then that's the way you want to go.
Martijn.
-
Hey Yael,
You should be able to. If it's being indexed, you can request removal via Bing Webmaster Tools and Google Search Console. You can also update your robots.txt file to disallow the page "http://www.robotstxt.org/robotstxt.html"
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unanswered questions in forums
What should be done with forum questions that go unanswered for a long time (i.e. year or longer)? Are these types of questions valuable content? Should we opt out of having these types of pages indexed? Since these pages are just one sentences doesn't seem like it is adding value to the site.
Intermediate & Advanced SEO | | nandaMesa0 -
Sort term product pages and fast indexing - XML sitemaps be updated daily, weekly, etc?
Hi everyone, I am currently working on a website that the XML sitemap is set to update weekly. Our client has requested that this be changed to daily. The real issue is that the website creates short term product pages (10-20 days) and then the product page URL's go 404. So the real problem is quick indexing not daily vs weekly sitemap. I suspect that daily vs weekly sitemaps may help solve the indexing time but does not completely solve the problem. So my question for you is how can I improve indexing time on this project? The real problem is how to get the product pages indexed and ranking before the 404 page shows u?. . Here are some of my initial thoughts and background on the project. Product pages are only available for 10 to 20 days (Auction site).Once the auction on the product ends the URL goes 404. If the pages only exist for 10 to 20 days (404 shows up when the auction is over), this sucks for SEO for several reasons (BTW I was called onto the project as the SEO specialist after the project and site were completed). Reason 1 - It is highly unlikely that the product pages will rank (positions 1 -5) since the site has a very low Domain Authority) and by the time Google indexes the link the auction is over therefore the user sees a 404. Possible solution 1 - all products have authorship from a "trustworthy" author therefore the indexing time improves. Possible solution 2 - Incorporate G+ posts for each product to improve indexing time. There is still a ranking issue here since the site has a low DA. The product might appear but at the bottom of page 2 or 1..etc. Any other ideas? From what I understand, even though sitemaps are fed to Google on a weekly or daily basis this does not mean that Google indexes them right away (please confirm). Best case scenario - Google indexes the links every day (totally unrealistic in my opinion), URL shows up on page 1 or 2 of Google and slowly start to move up. By the time the product ranks in the first 5 positions the auction is over and therefore the user sees a 404. I do think that a sitemap updated daily is better for this project than weekly but I would like to hear the communities opinion. Thanks
Intermediate & Advanced SEO | | Carla_Dawson0 -
Local Listing Question
We will be starting local SEO efforts on a medical practice that has 4 locations & 15 doctors each location (so 60 listings total). I will submit each doctor & each location to InfoGroup, LocalEze, Axciom & Factual. Also, I will only submit each location (not doctors) to Google. The problem I'm seeing is the fact that each listing would have the same exact phone number - it all goes to one main routing center. What kind of problems could come of this? Do we need a separate phone numbers for each of the four locations (at the very least)?
Intermediate & Advanced SEO | | JohnWeb120 -
Google Not Indexing XML Sitemap Images
Hi Mozzers, We are having an issue with our XML sitemap images not being indexed. The site has over 39,000 pages and 17,500 images submitted in GWT. If you take a look at the attached screenshot, 'GWT Images - Not Indexed', you can see that the majority of the pages are being indexed - but none of the images are. The first thing you should know about the images is that they are hosted on a content delivery network (CDN), rather than on the site itself. However, Google advice suggests hosting on a CDN is fine - see second screenshot, 'Google CDN Advice'. That advice says to either (i) ensure the hosting site is verified in GWT or (ii) submit in robots.txt. As we can't verify the hosting site in GWT, we had opted to submit via robots.txt. There are 3 sitemap indexes: 1) http://www.greenplantswap.co.uk/sitemap_index.xml, 2) http://www.greenplantswap.co.uk/sitemap/plant_genera/listings.xml and 3) http://www.greenplantswap.co.uk/sitemap/plant_genera/plants.xml. Each sitemap index is split up into often hundreds or thousands of smaller XML sitemaps. This is necessary due to the size of the site and how we have decided to pull URLs in. Essentially, if we did it another way, it may have involved some of the sitemaps being massive and thus taking upwards of a minute to load. To give you an idea of what is being submitted to Google in one of the sitemaps, please see view-source:http://www.greenplantswap.co.uk/sitemap/plant_genera/4/listings.xml?page=1. Originally, the images were SSL, so we decided to reverted to non-SSL URLs as that was an easy change. But over a week later, that seems to have had no impact. The image URLs are ugly... but should this prevent them from being indexed? The strange thing is that a very small number of images have been indexed - see http://goo.gl/P8GMn. I don't know if this is an anomaly or whether it suggests no issue with how the images have been set up - thus, there may be another issue. Sorry for the long message but I would be extremely grateful for any insight into this. I have tried to offer as much information as I can, however please do let me know if this is not enough. Thank you for taking the time to read and help. Regards, Mark Oz6HzKO rYD3ICZ
Intermediate & Advanced SEO | | edlondon0 -
XML Sitemaps for Message Boards / Forums - Best Practices?
I'm working with a message board that has been around for 10+ years and never taken SEO best practices into consideration. They recently started seeing mobile URLs show up in regular results, which they don't want. I'm recommending they implement multiple sitemaps to properly indicate to Google how to crawl the site and what to index. I've never dealt with a site this large so I'm not sure best practices. They have a HUGE community and new URLs are created every second. Doing a site: search returns "About 12,100,000" URLs. What are some best practices / the best way to approach sitemaps for a site of this size?
Intermediate & Advanced SEO | | MichaelWeisbaum0 -
Social Buttons Help SEO, 2 Questions...
Howdy Guys, I noticed a weird thing over the weekend - our main keyword has been hit pretty hard by penguin and we had dropped down to #79. On Friday I decided to change some on-page optimisation and changed the title tag and some tags. When I've ran my rank tracker this morning we have jumped up to #62... Has anyone else noticed just a simple change boosts rankings? Second Questions We took all our social buttons off the website back in January as no-body was using them but from a few recent reports I've seen having the buttons on the site help organic rankings... Is this true? Scott
Intermediate & Advanced SEO | | ScottBaxterWW0 -
What is the practical influence of priority in a sitemap?
I have a directory site with 1000s of entries. Will there be benefit to be gained from playing with various entries priorities in the sitemap? I was thinking I might give more priority to entries that have upgraded their directory entry. Thanks.
Intermediate & Advanced SEO | | flow_seo0 -
Multiple Keyword Research Questions, Help
Hello , I've been trying for several days to understand how keyword research works for a multi purpose website,I've read guides, articles even some chapters from the book" The Art of Seo" by O'Reilly and still no luck. It seems i can't wrap my head around keyword research,lets say I have a social gaming community website and I'm trying to rank it first on some low competition keywords + some long tail keywords.The website has functions like leaderboards, profiles,events, competitions,etc so it's not actually a news related website but it will have a blog. My website being on the games niche It would imply that I should target words that contain the word "Games" but this word generates millions of searches globally so ranking first its nearly impossible if the website is brand new. This made me pursue generic keywords formed with 2 / 3 words like fresh games, new games, mmorpg games, fps games,etc which still generate lets say 30.000 searches globally each. Due to the different areas of the website like latest game events,latest games competitions,etc I'm confused If i should pursue website specific keywords like latest games events, fresh games events, latest games competitions, upcoming games competitions but these too generate 30.000 global searches each,so... 0.should i use generic keywords or keywords that include site features? So let's say I decide to pursue generic "games" keywords,due to a high competition based on the keyword I decide to go a layer deeper and for the keyword "fresh games" I obtain keywords like** "fresh games 2011,top fresh games 2011, upcoming fresh games** " and thus building a list of 30 keywords that contain " fresh games".If i do this for the rest of the keywords: ** new games, mmorpg games, fps games,etc** I end up with a list of 10.000 keywords or more since each keyword generates other keywords. Is this the correct approach ? since generating 10.000 keywords sounds a lot and I'm getting the feeling that It's not how it supposed to be done,like were would I insert 10.000 keywords? So how do I know which keywords to pick and aim in order to try to get no.1 ranking? and why those? How many keywords should I use? and where should i put them? since it's not a news website so writing a lot of articles isn't an option. Should I focus on 2 words keywords with around 10.000-30.000 seaches or 2 words keywords + long tail keywords with less traffic like 100 - 5000? Is there a guide for the Keyword Analysis Tool since if i enter "fresh new games" i get a 39% keyword difficulty,is that hard to rank? and I don't know what all those color mean since some of them have higher numbers then others that are found at the top and how can i get beat a website that has has rank 10. So hopefully with your help & by some miracle I will finally be able to build a keyword list. Thank you !
Intermediate & Advanced SEO | | arching0