Hello,

Our company is international and we are looking to gain more traffic specifically from Europe. While I am aware that translating content into local languages, targeting local keywords, and gaining more European links will improve rankings, I am curious if it is worthwhile to have a company.eu domain in addition to our company.com domain.

Assuming the website's content and domain will be exactly the same, with the TLD (.eu vs .com) being the only change - will this add us benefit or will it hurt us by creating duplicate content - even if we create a separate GSC property for it with localized targeting and hreflang tags? Also - if we have multiple languages on our .eu website, can different paths have differing hreflangs?

IE: company.eu/blog/german-content German hreflang and company.eu/blog/Italian-content Italian hreflang.

I should note - we do not currently have an hreflang attribute set on our website as content has always been correctly served to US-based English speaking users - we do have the United States targeted in Google Search Console though.

It would be ideal to target countries by subfolder rather if it is just as useful. Otherwise, we would essentially be maintaining two sites.

Thanks!

It seemed to work. Hopefully the noindex is respected, thank you!

It looks like it is active. Thanks, John! Can you no-index an entire directory in GSC? I thought it was only per URL.

Hello,

I have a broken plugin creating hundreds of WP-Content directory pages being indexed by Google. I can not access the source code of these pages to add a noindex to them. The page URL's all have the plugin name within them.  In order to resolve the issue, I wrote a solution with javascript to dynamically add in a noindex tag to any URL containing the plugin name. Would this noindex be respected by Google and is there a way to immediately check that it is respected?

Currently, I can not delete the plugin due to issues with it's php.

If you would like to view the code: https://codepen.io/trodrick/pen/Gwwaej?editors=0010

Thanks!

I do agree, I may have to pass this off to someone with more backend experience than myself. In terms of plugins, are you aware of any that will allow you to add noindex tags to an entire folder?

Thanks!

Thank you for all your help. I added in a directive to 410 the pages in my htaccess as so: Redirect 410 /revslider*/. However, it does not seem to work.

Currently, I am using Options All -Indexes to 404 the URLs. Although I still remain worried as even though Google would not revisit a 410, could it still initially index it? This seems to be the case with my 404 pages - Google is actively indexing the new 404 pages that the broken plugin is producing.

As I can not seem to locate the directory in Cpanel, adding a noindex to them has been tough. I will look for a plugin that can dynamically add it based on folder structure because the URLs are still actively being created.

The ongoing creation of the URL's is the ultimate source of the issue, I expected that deleting the plugin would have resolved it but that does not seem to be the case.

Thank you for your response! I will certainly use the regex in my robots.txt and try to change my Htaccess directive to 410 the pages.

However, the issue is that a defunct plugin is randomly creating hundreds of these URL's without my knowledge, which I can not seem to access. As this is the case, I can't add a no-index tag to them.

This is why I manually de-indexed each page using the GSC removal tool and then blocked them in my robots.txt. My hope was that after doing so, Google would no longer be able to find the bad URL's.

Despite this, Google is still actively crawling & indexing new URL's following this path, even though they are blocked by my robots.txt (validated). I am unsure how these URL's even continue to be created as I deleted the plugin.

I had the idea to try to write a program with javascript that would take the status code and insert a no-index tag if the header returned a 404, but I don't believe this would even be recognized by Google, as it would be inserted dynamically. Ultimately, I would like to find a way to get the plugin to stop creating these URL's, this way I can simply manually de-index them again.

Thanks,

Hi All,

The site I am working on is built on Wordpress. The plugin Revolution Slider was downloaded. While no longer utilized, it still remained on the site for some time. This plugin began creating hundreds of URLs containing nothing but code on the page. I noticed these URLs were being indexed by Google. The URLs follow the structure: www.mysite.com/wp-content/uploads/revslider/templates/this-part-changes/

I have done the following to prevent these URLs from being created & indexed:

1. Added a directive in my Htaccess to 404 all of these URLs

2. Blocked /wp-content/uploads/revslider/ in my robots.txt

3. Manually de-inedex each URL using the GSC tool

4. Deleted the plugin

However, new URLs still appear in Google's index, despite being blocked by robots.txt and resolving to a 404. Can anyone suggest any next steps? I

Thanks!

Thank you for the help, Gaston!

Can I do so with:

Allow: *.jpg

Allow: *.png

Thanks, Gaston. I should have been more clear about what I am looking to do. I currently am having an indexation issue. Somehow, pages are being automatically generated by WordPress.

These pages are often .txt files of information or code from plugins, all beginning with /wp-content/uploads/ in their URL. I have been manually removing them from the index and would like to now have them be uncrawlable.

Best

Gaston,

Thanks for the fast reply! My images folder does follow that format, which is what makes me worrisome as we are blocking the wp-conent folder.

Thanks!

Hi Gaston,

I just wanted to follow up with you with one last question if possible. Would this allow my images and PDF's to be crawled & indexed still?

Thanks!

Awesome. Thanks, Gaston!

Thank you for the response. I'm still a little uncertain, does the version you wrote allow the bots to crawl the css and js as well?

Best

Hello,

I would like my website to remain crawlable to bots, but to block my wp content and media. Does the following robots.txt work? I worry that the * user agent may conflict with the others.

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/

User-agent: GoogleBot
Allow: /

User-agent: GoogleBot-Mobile
Allow: /

User-agent: GoogleBot-Image
Allow: /

User-agent: Bingbot
Allow: /

User-agent: Slurp
Allow: /

Using the htaccess I 404'd all the pages using "Options All -Indexes". Will this resolve the issue?

We use Worpdress as our CMS and do indeed use Yoast. We have never had an issue with /wp-content/ being indexed before, and I have been very conscientious about keeping our index clean.

Why I am confused is that this is an index of our wp-content, similar to a sitemap. I do not have robots.txt blocked for this as I do not know what is making the index.

Thanks!

Hi all,

I have suddenly noticed a massive jump in indexed pages. After performing a "site:" search, it was revealed that the sudden jump was due to the indexation of many pages beginning with the serp title "Index of /wp-content/uploads/" for many uploaded pieces of content & plugins.

This has appeared approximately one month after switching to https. I have also noticed a decline in Bing rankings. Does anyone know what is causing/how to fix this? To be clear, these pages are **not **normal /wp-content/uploads/ but rather "index of" pages, being included in Google.

Thank you.

Thanks for the help, Nicholas!

Sounds good, Ill give it a shot. Thank you guys!

Thanks for the input, Nicholas. This is what I was thinking, however, it seems that the blog post is now ranking for the last four days, and my solutions page isn't ranking at all for the keyword. Usually, the blog post would rank 1-2 days a week while the product page would rank the rest.

Would you still suggest de-optimizing the blog? Ranking for the keyword has been a months-long initiative, and I don't want to ruin my efforts.

Or should I wait and see if the product page begins ranking instead of the post again before de-optimizing the post?

Thanks for the response, I do have the keyword as the anchor text linking from my blog post to my product page. I don't know why it is when one ranks, the other does not, rather than alongside each other.

Would de-optimizing my blog post allow for my product page to rank all the time - or will it cause a lack of coverage when the blog post would otherwise rank?

Hello,

We are ranking for an acronym, which I understand can lead to fickle rankings. However, we have two pages ranking page one - two for the same keyword, but they do so in spite of each other.

By this I mean, one page will rank, while the other is nowhere to be found. It seems that the one page (a blog post) is more likely to rank on the weekends while the product page is more likely to rank on the weekdays.

I would like the product page to rank all the time, and to target another keyword with the blog post. Would removing the keyword from the blog post allow the product page to rank all the time - or would it lead to no pages ranking during times when the blog post would otherwise be ranking?

I should note the blog post has more external links and is not exactly optimized for the keyword, while the product page has more internal links and is optimized for the keyword.