Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

HTTP Status showing up in opensiteexplorer top pages as blocked by robot.txt file

Technical SEO

477

ReSEOlve last edited by

I am trying to find an answer to this question it has alot of url on this page with no data

when i go into the data source and search for noindex or robot.txt

but the site is visible in the search engines ?
1 Reply Last reply
Reply Quote 0
FedeEinhorn last edited by

Hi,

Would you mind sharing the URL?

Thanks!

Merry Christmas!
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

My WP website got attack by malware & now my website site:www.example.ca shows about 43000 indexed page in google.

Hi All My wordpress website got attack by malware last week. It affected my index page in google badly. my typical site:example.ca shows about 130 indexed pages on google. Now it shows about 43000 indexed pages. I had my server company tech support scan my site and clean the malware yesterday. But it still shows the same number of indexed page on google. Does anybody had ever experience such situation and how did you fixed it. Looking for help. Thanks FILE HIT LIST:
{YARA}Spam_PHP_WPVCD_ContentInjection : /home/example/public_html/wp-includes/wp-tmp.php
{YARA}Backdoor_PHP_WPVCD_Deployer : /home/example/public_html/wp-includes/wp-vcd.php
{YARA}Backdoor_PHP_WPVCD_Deployer : /home/example/public_html/wp-content/themes/oceanwp.zip
{YARA}webshell_webshell_cnseay02_1 : /home/example2/public_html/content.php
{YARA}eval_post : /home/example2/public_html/wp-includes/63292236.php
{YARA}webshell_webshell_cnseay02_1 : /home/example3/public_html/content.php
{YARA}eval_post : /home/example4/public_html/wp-admin/28855846.php
{HEX}php.generic.malware.442 : /home/example5/public_html/wp-22.php
{HEX}php.generic.cav7.421 : /home/example5/public_html/SEUN.php
{HEX}php.generic.malware.442 : /home/example5/public_html/Webhook.php
Technical SEO | | Chophel

0
Top domain(s) not at all showing up for branded keywords

Hey guys! Hope you're all doing well. I need your smart brains for a problem the company I work for is currently facing in the realm of SEO. They wanted to hire someone to find the problem, but I thought I would reach out to you guys, first, to at least try and narrow down the long list of things we could look at/investigate. I put the info (briefly) on 3 slides, and would really appreciate any and all insights you can offer me: https://docs.google.com/presentation/d/12d0YH7BNdCIpJ86d95uBOtOWphWQ3NB1kwDYeOzIt-U/edit#slide=id.g19145c9fa5_0_0 Thank you in advance!
Technical SEO | | michelledemaree

0
Multiple robots.txt files on server

Hi! I have previously hired a developer to put up my site and noticed afterwards that he did not know much about SEO. This lead me to starting to learn myself and applying some changes step by step. One of the things I am currently doing is inserting sitemap reference in robots.txt file (which was not there before). But just now when I wanted to upload the file via FTP to my server I found multiple ones - in different sizes - and I dont know what to do with them? Can I remove them? I have downloaded and opened them and they seem to be 2 textfiles and 2 dupplicates. Names: robots.txt (original dupplicate)
robots.txt-Original (original)
robots.txt-NEW (other content)
robots.txt-Working (other content dupplicate) Would really appreciate help and expertise suggestions. Thanks!
Technical SEO | | mjukhud

0
Can ht access file affect page load times

We have a large and old site. As we've transition from one CMS to another, there's been a need for create 301 redirects using our ht access file. I'm not a technical SEO person, but concerned that the size of our ht access file might be contributing source for long page download times. Can large ht access files cause slow page load times? Or is the coding of the 301 redirect a cause for slow page downloads? Thanks
Technical SEO | | ahw

1
Robots.txt crawling URL's we dont want it to

Hello We run a number of websites and underneath them we have testing websites (sub-domains), on those sites we have robots.txt disallowing everything. When I logged into MOZ this morning I could see the MOZ spider had crawled our test sites even though we have said not to. Does anyone have an ideas how we can stop this happening?
Technical SEO | | ShearingsGroup

0
Can I Disallow Faceted Nav URLs - Robots.txt

I have been disallowing /*? So I know that works without affecting crawling. I am wondering if I can disallow the faceted nav urls. So disallow: /category.html/? /category2.html/? /category3.html/*? To prevent the price faceted url from being cached: /category.html?price=1%2C1000
and
/category.html?price=1%2C1000&product_material=88 Thanks!
Technical SEO | | tylerfraser

0
Can I redirect when Google is showing these as 2 different pages?

Hi Guys, Google webmaster is showing 1000 duplicate title tags because its picking up our pages like this. How can I correct this? Please explain in detail please. Thank You Tim /store/ICICLES_NO_7_CLEAR_WITH_PINK_NUBBY//store/ICICLES_NO_7_CLEAR_WITH_PINK_NUBBY
Technical SEO | | fasctimseo

0
Many items with similar descriptions showing up as duplicate page titles

Hi, We are just a small company trying to understand this seo business. :O) I hope you will give us your input on how we look and how we can improve. www.bakerbay.com Each week I go down the seomoz crawl and change the duplicate page titles it increases the duplicate page content. Then the next week it seems to find other titles that have been fixed that are afluwie now. Please help. We are a bead company and alot of our products have a similar description but have different colors and items numbers. Could you please advise me on how to fix these errors and increase our ranking? All the Best, Beth
Technical SEO | | BakerBayBeadCo

0