VisitSweden indexing error
-
Hi all
Just got a new site up about weekend travel for VisitSweden, the official tourism office of Sweden. Everything went just fine except som issues with indexing.
The site can be found here at weekend.visitsweden.com/no/
For some weird reason the "frontpage" of the site does not get indexed. What I have done myself to find the issue:
- Added sitemaps.xml
- Configured and added site to webmaster tools
- Checked 301s so they are not faulty
By doing a simple site:weekend.visitsweden.com/no/ you can see that the frontpage is simple not in the index. Also by doing a cache:weekend.visitsweden.com/no/ I see that Google tries to index the page without the trailing /no/ for some reason.
http://webcache.googleusercontent.com/search?q=cache:http://weekend.visitsweden.com/no/
Any smart ideas to get this fixed or where to start looking?
All help greatly appreciated
Kind regards
Fredrik
-
Oh my God Fred!! the weekend sub-domain has been completely blocked from being crawled using a robots.txt file sitting in the root of the sub-domain.
http://weekend.visitsweden.com/robots.txt
User-agent: * Disallow: / Please remove the '/' from there **immediately**
-
Hi Fred,
I just copied my first response:
Here is your redirection setup:
http://weekend.visitsweden.com/ being redirected via 301 to
http://weekend.visitsweden.com/no being redirected via 301 to
http://weekend.visitsweden.com/no/
So, I would suggest you to remove the interim URL without the trailing slash after 'no'. Let the original homepage, http://weekend.visitsweden.com/ also be redirected to http://weekend.visitsweden.com/no/ (the one with trailing slash) via 301.
So your redirection setup should be as follows:
http://weekend.visitsweden.com/ - via 301 to - http://weekend.visitsweden.com/no/
Essentially, we are eliminating the redirection loop here. Please try this and post back.
Best regards,
Devanur Rafi
-
Hi
Again thanks for your quick response. Unfortunately we still have the same issue even though we have performed many checks and tests. Any more smart ideas on how this can be corrected?
Regards
Fredrik
-
Hi Fred,
Please wait for at least 2 weeks for the change to reflect in Google. This happens and depends on how popular your site is in terms of link profile, DA, PA etc..I still see "http://weekend.visitsweden.com/no" (without trailing slash) in Google's index. Let us wait for sometime. Nothing to worry about it.
-
Hi again
The weirdest this is that it does not seem to update. When I do a site:weekend.visitsweden.com/no/ the page is still nowhere to be found.
https://www.google.no/?gws_rd=ssl#q=site:weekend.visitsweden.com%2Fno%2F
Any ideas?
Again thanks
Fredrik
-
Hi Fred, now its perfect. It should soon reflect in Google and you will be able to see it in site: search. Good Luck my friend.
Best regards,
Devanur Rafi
-
Thanks for the great input! Have now tried to do the changes as per your suggestion.
Does it look better now?
Again thanks
Fredrik
-
Dear Fred,
Here is your redirection setup:
http://weekend.visitsweden.com/ being redirected via 301 to
http://weekend.visitsweden.com/no being redirected via 301 to
http://weekend.visitsweden.com/no/
So, I would suggest you to remove the interim URL without the trailing slash after 'no'. Let the original homepage, http://weekend.visitsweden.com/ also be redirected to http://weekend.visitsweden.com/no/ (the one with trailing slash) via 301.
So your redirection setup should be as follows:
http://weekend.visitsweden.com/ - via 301 to - http://weekend.visitsweden.com/no/
This should fix the issue. Essentially, we are eliminating the redirection loop here.
By the way, both the URLs, with and without trailing slash appear in Google when searched with the following queries:
Best regards,
Devanur Rafi
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What to do if lots of backend pages have been indexed by Google erroneously?
Hi Guys Our developer forgot to add a no index no follow tag on the pages he created in the back-end. So we have now ended up with lots of back end pages being indexed in google. So my question is, since many of those are now indexed in Google, so is it enough to just place a no index no follow on those or should we do a 301 redirect on all those to the most appropriate page? If a no index no follow is enough, that would create lots of 404 errors so could those affect the site negatively? Cheers Martin
Intermediate & Advanced SEO | | martin19700 -
Crawl and Indexation Error - Googlebot can't/doesn't access specific folders on microsites
Hi, My first time posting here, I am just looking for some feedback on a indexation issue we have with a client and any feedback on possible next steps or items I may have overlooked. To give some background, our client operates a website for the core band and a also a number of microsites based on specific business units, so you have corewebsite.com along with bu1.corewebsite.com, bu2.corewebsite.com. The content structure isn't ideal, as each microsite follows a structure of bu1.corewebsite.com/bu1/home.aspx, bu2.corewebsite.com/bu2/home.aspx and so on. In addition to this each microsite has duplicate folders from the other microsites so bu1.corewebsite.com has indexable folders bu1.corewebsite.com/bu1/home.aspx but also bu1.corewebsite.com/bu2/home.aspx the same with bu2.corewebsite.com has bu2.corewebsite.com/bu2/home.aspx but also bu2.corewebsite.com/bu1/home.aspx. Therre are 5 different business units so you have this duplicate content scenario for all microsites. This situation is being addressed in the medium term development roadmap and will be rectified in the next iteration of the site but that is still a ways out. The issue
Intermediate & Advanced SEO | | ImpericMedia
About 6 weeks ago we noticed a drop off in search rankings for two of our microsites (bu1.corewebsite.com and bu2.corewebsite.com) over a period of 2-3 weeks pretty much all our terms dropped out of the rankings and search visibility dropped to essentially 0. I can see that pages from the websites are still indexed but oddly it is the duplicate content pages so (bu1.corewebsite.com/bu3/home.aspx or (bu1.corewebsite.com/bu4/home.aspx is still indexed, similiarly on the bu2.corewebsite microsite bu2.corewebsite.com/bu3/home.aspx and bu4.corewebsite.com/bu3/home.aspx are indexed but no pages from the BU1 or BU2 content directories seem to be indexed under their own microsites. Logging into webmaster tools I can see there is a "Google couldn't crawl your site because we were unable to access your site's robots.txt file." This was a bit odd as there was no robots.txt in the root directory but I got some weird results when I checked the BU1/BU2 microsites in technicalseo.com robots text tool. Also due to the fact that there is a redirect from bu1.corewebsite.com/ to bu1.corewebsite.com/bu4.aspx I thought maybe there could be something there so consequently we removed the redirect and added a basic robots to the root directory for both microsites. After this we saw a small pickup in site visibility, a few terms pop into our Moz campaign rankings but drop out again pretty quickly. Also the error message in GSC persisted. Steps taken so far after that In Google Search Console, I confirmed there are no manual actions against the microsites. Confirmed there is no instances of noindex on any of the pages for BU1/BU2 A number of the main links from the root domain to microsite BU1/BU2 have a rel="noopener noreferrer" attribute but we looked into this and found it has no impact on indexation Looking into this issue we saw some people had similar issues when using Cloudflare but our client doesn't use this service Using a response redirect header tool checker, we noticed a timeout when trying to mimic googlebot accessing the site Following on from point 5 we got a hold of a week of server logs from the client and I can see Googlebot successfully pinging the site and not getting 500 response codes from the server...but couldn't see any instance of it trying to index microsite BU1/BU2 content So it seems to me that the issue could be something server side but I'm at a bit of a loss of next steps to take. Any advice at all is much appreciated!0 -
Why isn't Google indexing this site?
Hello, Moz Community My client's site hasn't been indexed by Google, although it was launched a couple of months ago. I've ran down the check points in this article https://moz.com/ugc/8-reasons-why-your-site-might-not-get-indexed without finding a reason why. Any sharp SEO-eyes out there who can spot this quickly? The url is: http://www.oldermann.no/ Thank you
Intermediate & Advanced SEO | | Inevo
INEVO, digital agency0 -
Sitemap Indexation
When we use HTML sitemap. Many a times i have seen that the sitemap itself gets mapped to keywords which it shouldn't have got to. So should we keep the HTML sitemap as No-Index, Follow or does anyone has a better solution that the sitemap doesn't show-up for other keyword terms that actually isn't representing this page.
Intermediate & Advanced SEO | | welcomecure0 -
Google Indexing of Images
Our site is experiencing an issue with indexation of images. The site is real estate oriented. It has 238 listings with about 1190 images. The site submits two version (different sizes) of each image to Google, so there are about 2,400 images. Only several hundred are indexed. Can adding Microdata improve the indexation of the images? Our site map is submitting images that are on no-index listing pages to Google. As a result more than 2000 images have been submitted but only a few hundred have been indexed. How should the site map deal with images that reside on no-index pages? Do images that are part of pages that are set up as "no-index" need a special "no-index" label or special treatment? My concern is that so many images that not indexed could be a red flag showing poor quality content to Google. Is it worth investing in correcting this issue, or will correcting it result in little to no improvement in SEO? Thanks, Alan
Intermediate & Advanced SEO | | Kingalan10 -
Do internal links from non-indexed pages matter?
Hi everybody! Here's my question. After a site migration, a client has seen a big drop in rankings. We're trying to narrow down the issue. It seems that they have lost around 15,000 links following the switch, but these came from pages that were blocked in the robots.txt file. I was wondering if there was any research that has been done on the impact of internal links from no-indexed pages. Would be great to hear your thoughts! Sam
Intermediate & Advanced SEO | | Blink-SEO0 -
Canonical Tags being indexed on paginated results?
On a website I'm working on which has a search feature with paginated results, all of the pages of the search results are set with a canonical tag back to the first page of the search results, however Google is indexing certain random pages within the result set. I can literally do a search in Google and find a deep page in the results, click on it and view source on that page and see that it has a canonical tag leading back to the first page of the set. Has anyone experienced this before? Why would Google not honor a canonical tag if it is set correctly? I've seen several SEO techniques for dealing with pagination, is there another solution that you all recommend?
Intermediate & Advanced SEO | | IrvCo_Interactive0 -
Sudden Index drop, but traffic increased?
Here are the numbers- Pages submitted on sitemap- About 18k Total Pages indexed on 12/30- About 250k Total Pages indexed on 1/6- About 81k We made no site changes in that week, why the sudden drop? Also why is total pages indexed so much higher than sitemap?
Intermediate & Advanced SEO | | EcommerceSite0