Robot.txt error

[Rubix]

I currently have this under my robot txt file:

User-agent: *
Disallow: /authenticated/
Disallow: /css/
Disallow: /images/
Disallow: /js/
Disallow: /PayPal/
Disallow: /Reporting/
Disallow: /RegistrationComplete.aspx

WebMatrix 2.0

On webmaster > Health Check > Blocked URL

I copy and paste above code then click on Test, everything looks ok but then logout and log back in then I see below code under Blocked URL:

User-agent: *

Disallow: /

WebMatrix 2.0

Currently, Google doesn't index my domain and i don't understand why this happening. Any ideas?

Thanks

Seda

[Rubix]

Thanks Irving, it worked

[irvingw]

Try to spider your site with this link checker tool

http://brokenlinkcheck.com/

bots cannot accept cookies and your site is requiring cookies to be enabled in order to be visited so Google cannot access the site because you are not allowing the visit without the cookie being dropped is most likely the issue.

Disable cookies on your browser and clear your cache and see what happens when you try to visit your site, are you blocked?

These discussions may possibly help

http://www.highrankings.com/forum/index.php/topic/3062-cookie-and-javascript/

http://stackoverflow.com/questions/5668681/seo-question-google-not-getting-past-cookies

[Rubix]

Thanks Irving, I need a little more help, I am not quite sure if I understand it. What is it that needs to be fixed here?

[Rubix]

I couldn't relay on SERPS as the website is old, it's been indexed for quite so i didn't think that SERP results would change that quick. I've been receiving the error since yesterday.

It's on SERPS today but would it be there tomorrow? The reason I am saying that is because when i change the Page Title, it doesnt get changed on SERPS instantly, it takes a day or so before i see the changes on SERPS.

[irvingw]

TECHNICAL ISSUE

It's your cookie policy blocking bots from spidering. Need to fix that at the server level. Easy fix!

http://www.positivecollections.co.uk/cookies-policy.aspx

Your robots.txt is fine.

[jesse-landry]

Okay. But that doesn't mean it isn't being indexed. Here's a fun test: Go to any page on your website and select a string of two or three sentences. Google it. Does the page come up in the SERPs?

(I did this to 3 pages on your site and it worked for all of them. Therefore, your site is being indexed.) Why do you need to Fetch?

[Rubix]

When I click on Fetch As Google, i get 'Denied by robots.txt'' error.

[jesse-landry]

That site is also being indexed. Again I ask, what makes you think it is not being indexed? (cause it is)

[Rubix]

When I click on Fetch As Google, i get 'Denied by robots.txt'' error.

@Jesse: That's the main website, we've got other URLs.Error appears on positivecollections.co.uk

[Rubix]

Thanks Irving,

www.positivecollections.co.uk is the url

I've tried to remove everything from the robot file and check again on webmaster, same thing happened  It's just blocking the main link

[jesse-landry]

Are you sure your site isn't being indexed?

Cause I went to your profile and if http://www.mtasolicitors.com/ is your site, then it is definitely being indexed.. What makes you think it isn't?

[irvingw]

Are you sure there is nothing else in your robots.txt - you can share the url if you like

You can delete this it's doing nothing and don't need to attempt to block bad bots

WebMatrix 2.0