Site Maps / Robots.txt etc
-
Hi everyone
I have setup a site map using a Wordpress pluggin: http://lockcity.co.uk/site-map/
Can you please tell me if this is sufficient for the search engines? I am trying to understand the difference between this and having a robots.txt - or do I need both?
Many thanks, Abi
-
Ahh I see thanks so much for your reply - is it best to get an XML site map rather than the one I have?
Thanks, Abi
-
A sitemap is for crawling your pages, either a user sitemap like you set up or a sitemap.xml file which is a streamlined way for Google to spider your site.
Robots.txt is used to direct the bots where NOT to go like if you don't want them crawling certain parts of your site like cgi-bin, js folder or images folder for example.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Word count and ranking/traffic
Hello All, Recently, a client of ours enlisted another firm to redesign their website, www.thepodhotel.com. The new site went live on February 20th. Since that date, we have seen a 30% drop in Google Organic traffic, year-over-year. Even a cursory glance at the site will tell you that there was quite a bit left on the table, from an SEO point of view. However, my question pertains to word count. The word count on the new site is quite a bit lower than the old site. How much value do you feel this has on rankings? Do you feel that, among the many other items that need to be addressed, we should be focusing on creating more copy? I appreciate all of your input. Frank
On-Page Optimization | | FrankSweeney0 -
Ratio follow/nofollow outbound links
Dear all, So far, I couldn't find any satisfying answer to my problem - I hope you might be able to help: Due to the fact that our website consists of user-generated content, we've got many many outbound links to other sites. Until now, it was possible to assign a follow-attribute to these links. In the ages of "pandas" however, we realised we had to limit this possibility as well as the amount of outbound follow links already published. My question now is whether there is some kind of rule of thumb as to what ratio of outbound follow and nofollow links is advisable. I'd appreciate any ideas or comments. Thanks a lot!
On-Page Optimization | | Mulle0 -
On Site Problem Caused Rankings To Drop?
I'm getting to the bottom of why my site dropped rankings on the 9th March. I noticed that in google there is a cached version of my site on 9th March at 8.37am which is when my rankings dissapeared. Presumably this is when google last crawled my site? I guess this means that google found something on the home page or on the site that it didn't like? I wonder if anyone can take a look and let me know if there's anything obvious. Could it be a duplicate content penalty as I have lots of categories pulling content from the same posts?
On-Page Optimization | | SamCUK0 -
Spanish version of site - best practice?
I need to create a Spanish version of an existing site. My idea was to have the Spanish content switch out the English content if the query string had something like ?l=es. It would also drop a cookie so that all other pages would switch out content as well. I do want the Spanish content to be indexed and rank in the search engines, though. I would include all of the Spanish versions (with the ?l=es) in the site map and link to them on every page with a link to the Spanish version. Does anyone have any experience with this? Is this a bad idea? Thanks! Tom
On-Page Optimization | | TomBristol0 -
Site Downtime - Will pages be reindexed?
I recently had site downtime of about 10 days for a site I'm working on which is about 6 months old. There were naturally many page not found errors in webmaster tools as google had tried to crawl the site during the downtime but I've noticed that google has now dropped some of the pages. Are they likely to be reindexed? Google has crawled some of my pages today but as yet the ones that have been dropped from the index haven't reappeared. (the site has been live again for a week). Should they reappear when the pages are next crawled?
On-Page Optimization | | SamCUK0 -
Site-wide keyword density
A colleague of mine was saying that he has been able to get top ranking for a high traffic term by using variations of that head term on multiple pages that are associated with the main page. For example,he would optimize a landing page for the high traffic word "Construction." He would then build pages under this landing page that are optimized for variations of this word: "Construction facts," "Industrial Construction Companies," "Construction Resource Allocator" etc. His theory is that the subpages add credibility with spiders that the root page is the best for that root page. This doesn't seem like it would work, but I'm curious as to what other people think.
On-Page Optimization | | EricVallee340 -
Site URL's
We are redeveloping our website, and have the option to amend URLs (with 301 redirects from old URL to new), so my question is: Would 'golfsite.com/golf-clubs' achieve superior rankings than 'golfsite.com/clubs' for the search term 'golf clubs' if all other factors were the same? Should the URL reflect the intended search term wherever possible?
On-Page Optimization | | swgolf1230 -
Sister Sites or Joint Family?
A large News Media Group has a Tv Channel, print newspaper, radio channel (for music primarly) and an online website that includes the newspaper content and other original content in different media. My question is, is it better to have independant websites for these different mediums or have all the content on one big website. Currently the newspaper and blog are online as one whereas the radio channel has its own website and the television has its own. So should we maintain sister sites and cross link to each other or have one big happy family under one house? Best, Rishad.
On-Page Optimization | | RishadShaikh590