Files blocked in robot.txt and seo
-
I use joomla and I have blocked the following in my robots.txt is there anything that is bad for seo ?
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/
Disallow: /mailto:myemail@myemail.com/
Disallow: /javascript:void(0)
Disallow: /.pdf
-
What you have there is just blocking rootdomain.com/javascript:void(0). Googlebot can execute and index JavaScript; you should not block it without a good reason. I'd let it read the JavaScript and see the submenus.
-
Thank you and is blocking javascript bad ? ( I was thinking about submenus )
-
If you don't want pages in those Disallowed directories to be indexed, then you're doing fine. These pages won't be able to be crawled, so, they won't be likely to appear in search results for any search engines.
The last three entries look fishy to me. I'd need to know what types of URLs you're trying to block to fix them. For the last one, if you're looking to block all pdfs on your site, the syntax would be Disallow: /*.pdf.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unique content for international SEO?
Hi Guys, We have a e-commerce store on generic top-level domain which has 1000s of products in US. We are looking to expand to aus, uk and canda using subfolders. We are going to implement hreflang tags. I was told by our SEO agency we need to make all the content between each page unique. This should be fine for cateogry/product listing pages. But they said we need to make content unique on product pages. If we have 1000 products, thats 4000 pages, which is a big job in terms of creating content. Is this necessary? What is the correct way to approach this, won't the hreflang tag be sufficent to prevent any duplicate content issues with product pages? Cheers.
Intermediate & Advanced SEO | | geekyseotools0 -
Alt tags seo boost
Do alt tags really give you a boost in terms of ranking or is it more for ranking in google images ? Thank you,
Intermediate & Advanced SEO | | seoanalytics0 -
WhoIs, SEO & Privacy
Is WHOIS data used by Google as a ranking signal? We had a website that had some bad SEO work done a while ago hence took a knock, If I use the same WHOIS data on a new site is that likely to cause an issue? Also I don't like the idea of providing too much information for privacy reasons, so have tended to stick to general email addresses and department names rather than actual personal information. Is that a bad approach?
Intermediate & Advanced SEO | | seoman100 -
SEO time
I wanto to be in the top of the google search. I am usiing a lot of SEO tools but... I have done it during one month. Do I have to wait more?
Intermediate & Advanced SEO | | CarlosZambrana0 -
Social Impacts on SEO? How to Do this?
I'm new in SEO and heard by one of my friend that social signals are important for SEO of a website. If people have shared a website's url on their twitter, then it will automatically get rank in google. Is that true and how google sees this social sharing? and how can I do this for my website?
Intermediate & Advanced SEO | | hammadrafique0 -
Best place to submit an SEO RFP? Anyone interested in 60 hours of SEO work?
I have a small SEO project (~ 60 hours of work) that I would like to get some help with. It is spread out over the span of 4 to 6 months (2 to 3 hours of work a week with the help of 10 - 15 support staff hours per week), and if it goes well there is an opportunity to extend the project through the rest of 2014. Does anyone here want to see the RFP or have any recommendations on where I can submit this request to get the maximum exposure? Thanks!
Intermediate & Advanced SEO | | pbhatt0 -
Is there a way to show random blocks of text to users without it affecting SEO? Cloaking for good?
My client has a pretty creative idea for his web copy. In the body of his page there will be a big block of text that contains random industry related terms but within that he will bold and colorize certain words that create a coherent sentence. Something to the effect of "cut through the noise with a marketing team that gets results". Get it? So if you were to read the paragraph word-for-word it would make no sense at all. It's basically a bunch of random words. He's worried this will affect his SEO and appear to be keyword stuffing to Google. My question is: Is there a way to block certain text on a webpage from search engines but show them to users? I guess it would be the opposite of cloaking? But it's still cloaking...isn't it? In the end we'll probably just make the block of text an image instead but I was just wondering if anyone has any creative solutions. Thanks!
Intermediate & Advanced SEO | | TheOceanAgency0 -
SEO Reco?
We're looking for a recommendation for a very good SEO agency that has experience with link building (white hat only). Any suggestions?
Intermediate & Advanced SEO | | BruceMillard0