Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Robots.txt issue for international websites

Intermediate & Advanced SEO

808

JinnatUlHasan last edited by

In Google.co.uk, our US based (abcd.com) is showing:

A description for this result is not available because of this site's robots.txt – learn more

But UK website (uk.abcd.com) is working properly. We would like to disappear .com result totally, if possible. How to fix it?

Thanks in advance.
1 Reply Last reply
Reply Quote 0
KeriMorgret last edited by

Can you share any information about your robots.txt?
1 Reply Last reply
Reply Quote 0
JinnatUlHasan @KeriMorgret last edited by

My main problem is in the homepage. Both host similar type of products and brands.

You may check the screenshot. Sorry, I had to blanked out the text.

Thanks in advance.

8BwQKcv
1 Reply Last reply
Reply Quote 0
KeriMorgret last edited by

Is it showing that for every page, or only some pages? If so, which types of pages? What's the contents of your robots.txt file for the US site?
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

SEO Best Practices regarding Robots.txt disallow

I cannot find hard and fast direction about the following issue: It looks like the Robots.txt file on my server has been set up to disallow "account" and "search" pages within my site, so I am receiving warnings from the Google Search console that URLs are being blocked by Robots.txt. (Disallow: /Account/ and Disallow: /?search=). Do you recommend unblocking these URLs? I'm getting a warning that over 18,000 Urls are blocked by robots.txt. ("Sitemap contains urls which are blocked by robots.txt"). Seems that I wouldn't want that many urls blocked. ? Thank you!!
Intermediate & Advanced SEO | | jamiegriz

0
International SEO Options?

Hi we currently have a site which is a example.com domain in the Australian market (we have geo-targeted to Australia within search console). We are looking to expand to United States. I have added the potential options down below, just wondering which one you guys think would be best from a SEO and practical standpoint? Or if there are other options i should consider? Option 1 The Australian domain is strong so this option takes this into consideration. Keep example.com (Australian) Add on: Sub-Directory for US Which would be: example.com/us/ In Search Console set the sub-folder to target US and also setup hreflang tags. Setup the US site on the sub-directory. Option 2 Add sub-folders for both Aus and US example/au/ (Australian)
example/us/ (United States) Setup hreflang targeting. Cons
Need to set up redirects for the current site to new location which is .com/au/ might also see drop in performance due to redirects. Cheers.
Intermediate & Advanced SEO | | jaynamarino

0
International Site Migration

Hi guys, In the process of launching internationally ecommerce site (Magento CMS) for two different countries (Australia and US). Then later on expand to other countries like the UK, Canada, etc. The plan is for each country will have its own sub-folder e.g. www.domain.com/us, www.domain.com.au/au, www.domain.com.au/uk A lot of the content between these English based countries are the same. E.g. same product descriptions.
So in order to prevent duplication, from what I’ve read we will need to add Hreflang tags to every single page on the site? So for: Australian pages: United States pages: Just wanted to make sure this is the correct strategy (will hreflang prevent duplicate content issues?) and anything else i should be considering? Thankyou, Chris
Intermediate & Advanced SEO | | jayoliverwright

0
Linking to External Websites?

Is it good to link external websites from every page. Since, the on-page grader shows there should be one link pointing to an external source. I have a website that can point to an external website from every page using the brand name of the specific site like deal sites do have. Is it worth having external link on every page, of-course with a no-follow tag?
Intermediate & Advanced SEO | | welcomecure

0
SEO transfer to new website

My website currently has some strong SEO and i will be re-developing the website on a wordpress platform...which will change many of the existing URL. Will this affect the current pages that are well indexed in Google? Does using Wordpress or changing the URL extension (.html to .php) make a difference? If i want to make a clean transition without effecting our existing SEO...what are some essential steps i need to take? Example. Current page is www.mydomain.com/name.html .... and the new URL would be www.mydomain.com/product/name.php Thanks
Intermediate & Advanced SEO | | Souk

0
Block in robots.txt instead of using canonical?

When I use a canonical tag for pages that are variations of the same page, it basically means that I don't want Google to index this page. But at the same time, spiders will go ahead and crawl the page. Isn't this a waste of my crawl budget? Wouldn't it be better to just disallow the page in robots.txt and let Google focus on crawling the pages that I do want indexed? In other words, why should I ever use rel=canonical as opposed to simply disallowing in robots.txt?
Intermediate & Advanced SEO | | YairSpolter

0
Google does not favour php websites?

Hi there. An SEO company recently told me that google does not favour php development? This seems rather sketchy, I have not read that google doesn't favour this anywhere, did I just miss that part of SEO or are these guys blowing a little smoke?
Intermediate & Advanced SEO | | ProsperoDigital

1
Have I missed anthing On this website?

Hello SEO MOzzers, I am today wanting your feedback on a site that I recently went live with. My Google rankings for the main keywords are doing very well considering the site has been live for 3 weeks now. I of course have a list of items that i'm still working on, completing meta description tags, title tags, adding copy content to category pages, updating h1 tags, working on our backlinking campaign, etc. Is there anthing any of you SEO Mozzers can see that I may have missed? the web domain is www.carbodypanels4u.co.uk Keywords "body Panels"
"car body panels"
"Car Make body panels"
"Car make Model body panel" Im sure you get the gist of the keyword structure. Your feedback is much appreciated as always. shiv
Intermediate & Advanced SEO | | seohive-222720

0