Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Robots.txt not working?

2

4

965

Log in to reply

levalencia1 last edited by
Hello

This is my robots.txt file

http://www.theprinterdepo.com/Robots.txt

However I have 8000 warnings on my dashboard like this:4

What am I missing on the file¿

Crawl Diagnostics Report

On-Page Properties

<dl>

<dt>Title</dt>

<dd>Not present/empty</dd>

<dt>Meta Description</dt>

<dd>Not present/empty</dd>

<dt>Meta Robots</dt>

<dd>Not present/empty</dd>

<dt>Meta Refresh</dt>

<dd>Not present/empty</dd>

</dl>

URL: http://www.theprinterdepo.com/catalog/product_compare/add/product/100/uenc/aHR0cDovL3d3dy50aGVwcmludGVyZGVwby5jb20vaHAtbWFpbnRlbmFjZS1raXQtZm9yLTQtbGo0LWxqNS1mb3ItZXhjaGFuZ2UtcmVmdWJpc2hlZA,,/

0 Errors
- No errors found!
1 Warning
- 302 (Temporary Redirect)
  
  Found about 5 hours ago
  
  <a class="more">Read More</a>
1 Reply Last reply
Reply Quote 0
levalencia1 last edited by

anyone in here? what should I do to fix this?
1 Reply Last reply
Reply Quote 0
levalencia1 @SlavaRybalka last edited by

No I havent tried but I will.

I use magento commerce which is a great ecommerce application
1 Reply Last reply
Reply Quote 0
SlavaRybalka last edited by

1. Have you tried it without product_compare?

2. Bad CMS?
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Rel Canonicals not working properly.

We recently implemented rel=canonicals on a few of our pages to prevent query parameters from showing up in the SERPs. The two pages we added the tags to are no longer ranking. The pages used to rank very well for branded terms such as "morningstar direct" and "morningstar sustainability", but now don't show up at all. When you search for the urls specifically, for example "products/direct morningstar" the query parameter is still indexing. Does anyone know why this might be or what we can do to fix this issue? The two pages are www.morningstar.com/products/direct and https://www.morningstar.com/company/sustainability
Technical SEO | | jmigdal

0
About how google works

Hey guyz,
I want to ask a basic question. If I search for Larry Page lets say.
I think google look for it's index for word larry and page distinct.
And mix it up. But the question ;
Can google show a result which only Larry exist on the page but any of the synonym or the stem of the Page not exist .
If it can happen how this page can be showned in larry page query. Thank you.
Technical SEO | | atakala

0
301 redirect not working

Hi there! I have recently moved a domain that has been indexed by google and setup redirects so that it forwards to the new domain. It seems like the only redirect that actually is working is the canonical and main domain but every other page and or page nested within a folder are not working. Here is an example of some of the redirects. Am I doing this wrong? It seems to be going to the new domain but can't find the actual pages.... RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !agoodsweep.com$ [NC]
RewriteRule ^(.*)$ http://agoodsweep.com/$1 [L,R=301]
redirect 301 woodstoveservicerepair.html http://agoodsweep.com/woodstoveservicerepair/
redirect 301 /westchesterchimney.html http://agoodsweep.com/west-chester-chimney/ Thanks in advance for any help!!
Technical SEO | | twotd

0
Wordpress Robots.txt Sitemap submission?

Alright, my question comes directly from this article by SEOmoz http://www.seomoz.org/learn-seo/r... Yes, I have submitted the sitemap to google, bing's webmaster tools and and I want to add the location of our site's sitemaps and does it mean that I erase everything in the robots.txt right now and replace it with? <code>User-agent: * Disallow: Sitemap: http://www.example.com/none-standard-location/sitemap.xml</code> <code>???</code> because Wordpress comes with some default disallows like wp-admin, trackback, plugins. I have also read this, but was wondering if this is the correct way to add sitemap on Wordpress Robots.txt. [http://www.seomoz.org/q/removing-...](http://www.seomoz.org/q/removing-robots-txt-on-wordpress-site-problem) I am using Multisite with Yoast plugin so I have more than one sitemap.xml to submit Do I erase everything in Robots.txt and replace it with how SEOmoz recommended? hmm that sounds not right. like <code> <code>
<code>User-agent: *
Disallow: </code> Sitemap: http://www.example.com/sitemap_index.xml</code> <code>``` Sitemap: http://www.example.com/sub/sitemap_index.xml ```</code> <code>?????????</code> ```</code>
Technical SEO | | joony2008

0
Warnings for blocked by blocked by meta-robots/meta robots Nofollow...how to resolve?

Hello, I see hundreds of notices for blocked by meta-robots/meta robots nofollow and it appears it is linked to the comments on my site which I assume I would not want to be crawled. Is this the case and these notices are actually a positive thing? Please advise how to clear them up if these notices can be potentially harmful for my SEO. Thanks, Talia
Technical SEO | | M80Marketing

0
Restricted by robots.txt does this cause problems?

I have restricted around 1,500 links which are links to retailers website and links that affiliate links accorsing to webmaster tools Is this the right approach as I thought it would affect the link juice? or should I take the no follow out of the restricted by robots.txt file
Technical SEO | | ocelot

0
Should search pages be disallowed in robots.txt?

The SEOmoz crawler picks up "search" pages on a site as having duplicate page titles, which of course they do. Does that mean I should put a "Disallow: /search" tag in my robots.txt? When I put the URL's into Google, they aren't coming up in any SERPS, so I would assume everything's ok. I try to abide by the SEOmoz crawl errors as much as possible, that's why I'm asking. Any thoughts would be helpful. Thanks!
Technical SEO | | MichaelWeisbaum

0
Using a table with tabs to display information on website, work for seo?

When displaying data using a table, and a tab format to seperate different options, for example http://www.mousetraining.co.uk/ms-training/microsoft-excel-training-courses.html - under Standard Excel Training Course Levels / Training Details / Locations / Schedule - at the bottom of the page. Would search engines pick up the keywords from each of the tabs, or are they hidden?? Thanks
Technical SEO | | jpc1004

0