Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Robots blocked by pages webmasters tools

Intermediate & Advanced SEO

2

7

637

Log in to reply

mihoreis last edited by
- a mistake made in software. How can I solve the problem quickly? help me.
XTRjH
1 Reply Last reply
Reply Quote 0
mihoreis @GastonRiera last edited by

Ok,

Updated.

I need to wait 1 week then

Thank You, Gaston
1 Reply Last reply
Reply Quote 0
Gaston Riera @mihoreis last edited by

That sitemap line place it at the end of the file.
like this:

User-agent: *
Disallow:

Sitemap: https://site.com/sitemap.xml

Best luck!
1 Reply Last reply
Reply Quote 1
mihoreis @GastonRiera last edited by

OK,

I did,

can you check from the picture.

Thank You.

zZd9P
1 Reply Last reply
Reply Quote 0
Gaston Riera @mihoreis last edited by

its better to set it as I've discribed before:

User-agent: *
Disallow:

It would take over a week, depending in scale of the website.
1 Reply Last reply
Reply Quote 1
mihoreis @GastonRiera last edited by

Yes,

in the shape of the photo,

is it correct? re-scanning takes a long time?

c4ZCy
1 Reply Last reply
Reply Quote 0
Gaston Riera last edited by

Hello there!

Have you identified all pages that you want to be blocked?
If you just do not want any page to be blocked, just edit you robots.txt as:

User-agent: *
Disallow:

For more information, check these resources:
Cómo bloquear URLs con robots.txt - Google Webmasters
About /robots.txt - robotstxt.org

Best Luck!
GR
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

May integrating my main category page in the index page improve my ranking of main category keyword?

90% of our sales are made with products in one of our product categories.
A search for main category keyword returns our root domain index page in google, not the category page.
I was wondering whether integrating the complete main category directly in the index page of the root domain and this way including much more relevant content for this main category keyword may have a positive impact on our google ranking for the main category keyword. Any thoughts?
Intermediate & Advanced SEO | | lcourse

1
Pages Disappearing from Search

Hi, We have had a strongly ranking site since 2004. Over the past couple of days, our Google traffic has dropped by around 20% and some of our strong pages are completely disappearing from the rankings. They are still indexed, but having ranked number 1 are nowhere to be found. A number of pages still remain intact, but it seems they are increasingly disappearing. Where should we start to try and find out what is happening? Thanks
Intermediate & Advanced SEO | | simonukss

0
How to handle broken links to phantom pages appearing in webmaster tools

Hi,Would love to hear different experiences and thoughts on this one. We have a site that is plagued with 404's in the Webmaster Tools. A significant number of them have never existed, for instance affiliates have linked to them with the wrong URL or scraper sites have linked to them with a truncated version of the URL and an ellipsis eg; /my-nonexistent... What's the best way to handle these? If we do nothing and mark as fixed, they reappear in the broken links report. If we 301 redirect and mark as fixed they reappear. We tried 410 (gone forever) and marking as fixed; they re-appeared. We have a lot of legacy broken links and we would really like to clean up our WMT broken link profile - does anyone know of a way we can make these links to non extistent pages disappear once and for all? Many thanks in advance!
Intermediate & Advanced SEO | | dancape

0
Why Is Google Webmaster Tools Pulling Zero Keyword Data?

I just linked a Google Webmaster Tools account to Google Analytics for a client, and Search Engine Optimization reports are showing up in Google Analytics as enabled, but there is zero keyword data, landing page data, etc., in the reports themselves. Has anyone encountered this?
Intermediate & Advanced SEO | | yoursearchteam

0
What exactly is an impression in Google Webmaster Tools search queries with the image filter turned on?

Is it when someone does an image search? Or does it count a regular search that has images in it? On an image search does the picture actually have to be viewed on the screen or can it be below in the infinite scroll?
Intermediate & Advanced SEO | | EcommerceSite

0
Google Webmaster Remove URL Tool

Hi All, To keep this example simple.
You have a home page. The home page links to 4 pages (P1, P2, P3, P4). ** Home page**
P1 P2 P3 P4 You now use Google Webmaster removal tool to remove P4 webpage and cache instance. 24 hours later you check and see P4 has completely disappeared. You now remove the link from the home page pointing to P4. My Question
Does Google now see only pages P1, P2 & P3 and therefore allocate link juice at a rate of 33.33% each. Regards Mark
Intermediate & Advanced SEO | | Mark_Ch

0
Amount of pages indexed for classified (number of pages for the same query)

I've notice that classified usually has a lots of pages indexed and that's because for each query/kw they index the first 100 results pages, normally they have 10 results per page. As an example imagine the site www.classified.com, for the query/kw "house for rent new york" there is the page www.classified.com/houses/house-for-rent-new-york and the "index" is set for the first 100 SERP pages, so www.classified.com/houses/house-for-rent-new-york www.classified.com/houses/house-for-rent-new-york-1 www.classified.com/houses/house-for-rent-new-york-2 ...and so on. Wouldn't it better to index only the 1st result page? I mean in the first 100 pages lots of ads are very similar so why should Google be happy by indexing lots of similar pages? Could Google penalyze this behaviour? What's your suggestions? Many tahnks in advance for your help.
Intermediate & Advanced SEO | | nuroa-246712

0
Landing Page - Home Page redesign SEO factor question - Serious concern.

Hi Folks, I'm considering making a big change to our website and really need some expert advise. Will we lose ranking if we do what I propose? Our site www.meninkilts.com, needs to split users/clients by "Commercial" and "Residential" so we can message/market completely differently to each client. We are considering doing this structure: Landing Page | | Commercial Homepage Residential Homepage Right now we rank well, for our keywords like "Window Cleaning cityname" but are worried that adding a landing page, and splitting our site to two homepages will effect seo (ie: a landing page would only have two buttons: one for commercial and one for residential). What would be the best way to handle this. Looking forward to your insights! Cheers Brent
Intermediate & Advanced SEO | | MenInKilts

0