Site Maps / Robots.txt etc
-
Hi everyone
I have setup a site map using a Wordpress pluggin: http://lockcity.co.uk/site-map/
Can you please tell me if this is sufficient for the search engines? I am trying to understand the difference between this and having a robots.txt - or do I need both?
Many thanks, Abi
-
Ahh I see thanks so much for your reply - is it best to get an XML site map rather than the one I have?
Thanks, Abi
-
A sitemap is for crawling your pages, either a user sitemap like you set up or a sitemap.xml file which is a streamlined way for Google to spider your site.
Robots.txt is used to direct the bots where NOT to go like if you don't want them crawling certain parts of your site like cgi-bin, js folder or images folder for example.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Meta Robots information
Hi, I have a question about the Meta Robots information Accoarding to the Moz bar, our page uses the meta robots noodp and noydir. Our competitor uses
On-Page Optimization | | AdoBike
INDEX,FOLLOW I read that noodp and noydir are dated and not used anymore. Is it wise to use INDEX FOLLOW instead for better SEO? Thanks in advance!1 -
Best site Template, Structure, etc. for SEO
If I were to spin up a new site what do people recommend as the best template, services, etc. Do you have an example of the perfect structure, I want to point my team to an example page and say - This is perfect, do this but for our product (structure, content amount, etc) Thank you,
On-Page Optimization | | Jamesmcd030 -
Expert to Improve Site Speed Needed
Hey, I am looking for an expert who can improve the speed performance for my website NaturalSociety.com. Please please email me at mike@naturalsociety.com if you or your company works to improve website speed performance, or if you can recommend someone. Thanks in advance, -Mike
On-Page Optimization | | naturalsociety0 -
Do https sites rank as well as http sites?
2 Questions: Question 1 - We currently have our entire site running on https (the http pages 301-redirect to the https versions). Assuming that the https pages load as quickly as the http versions, is it a problem that the entire site is https? The only official answer I've been able to find is this 2011 video where Matt Cutts basically says "I don't know" - http://www.youtube.com/watch?v=xeFo4ytOk8M Question 2 - Is there any problem with having half our site running on https only (with the http pages redirected), and the other half (our blog) running on http only (with all https blog pages redirected to the http versions)? Thanks in advance for any input! Justin
On-Page Optimization | | JustinClark0 -
Which Blog Platform to link to an eCommerce site is best?
I just hired a content writer to blog on my site, but I want to make sure I have the right blog set up properly before doing so. I currently have my blog on my own domain. http://alturl.com/ixd7p It's a pretty crappy blog, to be quite frank. (The link to the blog is in the footer). It doesn't allow me to change category titles so it's throwing duplicate content - not good. I am seriously considering getting a self hosted Wordpress blog and linking that to my site instead at 3dcart- so it will be blog.domainname.comMy CONCERN.... I always heard that it's best to have the blog right right on the eCommerce site (just as I have it now) because it keeps feeding the domain fresh content. If I have a self hosted Wordpress blog and have it linked to my site, will it still feed my site fresh content?
On-Page Optimization | | tutugirl0 -
How to design a site map page for users (not for Google)
I would like to design a site map for my visitors so they can have a quick view on the whole content of the website. 2 questions : 1 - is this kind of site map can help in terms of SEO ? 2 - if so, what are the best practices to design it ? Thanks in advance.
On-Page Optimization | | betadvisor0 -
Monitor site loading speeds
We all know that site loading speed is important for google ( and getting more important), and currently we are considering moving to a dedicated server. But I would like to measure pur current loading times and compare them to our competitors. I know that webmaster tools shows you your loading times, but its seems a bit random with its results, and I know analytics also now records speed, but the sample sizes seems to be too low. When I googled website load times, I got a load of different free tools but they seem to give conflicting results e.g one said that our avg kb load time was 0.02sec and our competitor was 0.01, but a different site says our sites loads in 1.7 secs and our competitor loads in 11sec (our home pages have similar total file size for our homepage. ) Also what would be useful would be a way to monitor of website loading speed to see when it slows down, so we can figure out why. Does anyone know of any good reliable programs or sites that does this?
On-Page Optimization | | eunaneunan0 -
URL with two forward slashes //
We have a potential client with a URL structure in this fashion: http://www.site-url.com//cpage/page.html pretty strange, right? my question is: How bad are the 2 forward slashes // for SEO? How bad is it to have that extra layer of /cpage in the URL? this doesn't appear to serve any other purpose than making the URL longer than necessary.
On-Page Optimization | | Motava0