Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Robots blocked by pages webmasters tools

Intermediate & Advanced SEO

2

7

640

Log in to reply

mihoreis last edited by
- a mistake made in software. How can I solve the problem quickly? help me.
XTRjH
1 Reply Last reply
Reply Quote 0
mihoreis @GastonRiera last edited by

Ok,

Updated.

I need to wait 1 week then

Thank You, Gaston
1 Reply Last reply
Reply Quote 0
Gaston Riera @mihoreis last edited by

That sitemap line place it at the end of the file.
like this:

User-agent: *
Disallow:

Sitemap: https://site.com/sitemap.xml

Best luck!
1 Reply Last reply
Reply Quote 1
mihoreis @GastonRiera last edited by

OK,

I did,

can you check from the picture.

Thank You.

zZd9P
1 Reply Last reply
Reply Quote 0
Gaston Riera @mihoreis last edited by

its better to set it as I've discribed before:

User-agent: *
Disallow:

It would take over a week, depending in scale of the website.
1 Reply Last reply
Reply Quote 1
mihoreis @GastonRiera last edited by

Yes,

in the shape of the photo,

is it correct? re-scanning takes a long time?

c4ZCy
1 Reply Last reply
Reply Quote 0
Gaston Riera last edited by

Hello there!

Have you identified all pages that you want to be blocked?
If you just do not want any page to be blocked, just edit you robots.txt as:

User-agent: *
Disallow:

For more information, check these resources:
Cómo bloquear URLs con robots.txt - Google Webmasters
About /robots.txt - robotstxt.org

Best Luck!
GR
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

When serving a 410 for page gone, should I serve an error page?

I'm removing a bunch of old & rubbish pages and was going to serve 410 to tell google they're gone (my understanding is it'll get them out of the index a bit quicker than a 404). I should still serve an error page though, right? Similar to a 404. That doesn't muddy the "gone" message that I'm giving Google? There's no need to 410 and die?
Intermediate & Advanced SEO | | HSDOnline

0
How long to re-index a page after being blocked

Morning all! I am doing some research at the moment and am trying to find out, just roughly, how long you have ever had to wait to have a page re-indexed by Google. For this purpose, say you had blocked a page via meta noindex or disallowed access by robots.txt, and then opened it back up. No right or wrong answers, just after a few numbers 🙂 Cheers, -Andy
Intermediate & Advanced SEO | | Andy.Drinkwater

0
Will Using Attributes For Landing Pages In Magento Dilute Page Rank?

Hello Mozzers! We have an ecommerce site built on Magento. We would like to use attribute filters in our layered navigation for landing page purposes. Each page will have a unique URL, Meta Title and Meta Description. For example: URL: domain.com/art/abstract (category is Art, attribute is Abstract) Title: Abstract Art For Sale Meta: Blah Blah Blah Currently these attribute pages are not being indexed by google as they are set in google parameters. We would like to edit google parameters to start indexing some of the attribute filters that users search for, so they can be used as landing pages. Does anyone have experience with this? Is this a good idea? What are the consequences? Will this dilute Page Rank? Could this destroy the world? Cheers! MozAddict
Intermediate & Advanced SEO | | MozAddict

0
Duplicate content within sections of a page but not full page duplicate content

Hi, I am working on a website redesign and the client offers several services and within those services some elements of the services crossover with one another. For example, they offer a service called Modelling and when you click onto that page several elements that build up that service are featured, so in this case 'mentoring'. Now mentoring is common to other services therefore will feature on other service pages. The page will feature a mixture of unique content to that service and small sections of duplicate content and I'm not sure how to treat this. One thing we have come up with is take the user through to a unique page to host all the content however some features do not warrant a page being created for this. Another idea is to have the feature pop up with inline content. Any thoughts/experience on this would be much appreciated.
Intermediate & Advanced SEO | | J_Sinclair

0
Home Page or Internal Page

I have a website that deals with personalized jewelry, and our main keyword is "Name Necklace".
3 mounth ago i added new page: http://www.onecklace.com/name-necklaces/ And from then google index only this page for my main keyword, and not our home page.
Beacuase the page is new, and we didn't have a lot of link to it, our rank is not so well. I'm considering to remove this page (301 to home page), beacause i think that if google index our home page for this keyword it will be better. I'm not sure if this is a good idea, but i know that our home page have a lot of good links and maybe our rank will be higher. Another thing, because google index this internal page for this keyword, it looks like our home page have no main keyword at all. BTW, before i add this page, google index our main page with this keyword. Please advise... U5S8gyS.png j50XHl4.png
Intermediate & Advanced SEO | | Tiedemann_Anselm

0
Infographics and articles on the same page

Hi i'm currently designing an infographic and the infographic is based on a article my writer had created. I was thinking of ways in which i can add the article and infographic so they complement each other. Obviously the infographic is more in-depth then the article as it contains much more information. The infographic is designed to go viral. I was thinking of putting the info graphic on the top of the page and the written content below it. This way the person looking at the infographic can scroll down to find the more in-depth written discussion/article on the topic. Also from a SEO perspective, the search engines can index the written content (as it won't be able to index the the infographic since it's a image. What do you guys think is the best approach for this situation? Regards, Matt
Intermediate & Advanced SEO | | Mattcarter08

0
Amount of pages indexed for classified (number of pages for the same query)

I've notice that classified usually has a lots of pages indexed and that's because for each query/kw they index the first 100 results pages, normally they have 10 results per page. As an example imagine the site www.classified.com, for the query/kw "house for rent new york" there is the page www.classified.com/houses/house-for-rent-new-york and the "index" is set for the first 100 SERP pages, so www.classified.com/houses/house-for-rent-new-york www.classified.com/houses/house-for-rent-new-york-1 www.classified.com/houses/house-for-rent-new-york-2 ...and so on. Wouldn't it better to index only the 1st result page? I mean in the first 100 pages lots of ads are very similar so why should Google be happy by indexing lots of similar pages? Could Google penalyze this behaviour? What's your suggestions? Many tahnks in advance for your help.
Intermediate & Advanced SEO | | nuroa-246712

0
Robots.txt is blocking Wordpress Pages from Googlebot?

I have a robots.txt file on my server, which I did not develop, it was done by the web designer at the company before me. Then there is a word press plugin that generates a robots.txt file. How Do I unblock all the wordpress pages from googlebot?
Intermediate & Advanced SEO | | ENSO

0