VisitSweden indexing error
-
Hi all
Just got a new site up about weekend travel for VisitSweden, the official tourism office of Sweden. Everything went just fine except som issues with indexing.
The site can be found here at weekend.visitsweden.com/no/
For some weird reason the "frontpage" of the site does not get indexed. What I have done myself to find the issue:
- Added sitemaps.xml
- Configured and added site to webmaster tools
- Checked 301s so they are not faulty
By doing a simple site:weekend.visitsweden.com/no/ you can see that the frontpage is simple not in the index. Also by doing a cache:weekend.visitsweden.com/no/ I see that Google tries to index the page without the trailing /no/ for some reason.
http://webcache.googleusercontent.com/search?q=cache:http://weekend.visitsweden.com/no/
Any smart ideas to get this fixed or where to start looking?
All help greatly appreciated
Kind regards
Fredrik
-
Oh my God Fred!! the weekend sub-domain has been completely blocked from being crawled using a robots.txt file sitting in the root of the sub-domain.
http://weekend.visitsweden.com/robots.txt
User-agent: * Disallow: / Please remove the '/' from there **immediately** -
Hi Fred,
I just copied my first response:
Here is your redirection setup:
http://weekend.visitsweden.com/ being redirected via 301 to
http://weekend.visitsweden.com/no being redirected via 301 to
http://weekend.visitsweden.com/no/
So, I would suggest you to remove the interim URL without the trailing slash after 'no'. Let the original homepage, http://weekend.visitsweden.com/ also be redirected to http://weekend.visitsweden.com/no/ (the one with trailing slash) via 301.
So your redirection setup should be as follows:
http://weekend.visitsweden.com/ - via 301 to - http://weekend.visitsweden.com/no/
Essentially, we are eliminating the redirection loop here. Please try this and post back.
Best regards,
Devanur Rafi
-
Hi
Again thanks for your quick response. Unfortunately we still have the same issue even though we have performed many checks and tests. Any more smart ideas on how this can be corrected?
Regards
Fredrik
-
Hi Fred,
Please wait for at least 2 weeks for the change to reflect in Google. This happens and depends on how popular your site is in terms of link profile, DA, PA etc..I still see "http://weekend.visitsweden.com/no" (without trailing slash) in Google's index. Let us wait for sometime. Nothing to worry about it.
-
Hi again
The weirdest this is that it does not seem to update. When I do a site:weekend.visitsweden.com/no/ the page is still nowhere to be found.
https://www.google.no/?gws_rd=ssl#q=site:weekend.visitsweden.com%2Fno%2F
Any ideas?
Again thanks
Fredrik
-
Hi Fred, now its perfect. It should soon reflect in Google and you will be able to see it in site: search. Good Luck my friend.
Best regards,
Devanur Rafi
-
Thanks for the great input! Have now tried to do the changes as per your suggestion.
Does it look better now?
Again thanks
Fredrik
-
Dear Fred,
Here is your redirection setup:
http://weekend.visitsweden.com/ being redirected via 301 to
http://weekend.visitsweden.com/no being redirected via 301 to
http://weekend.visitsweden.com/no/
So, I would suggest you to remove the interim URL without the trailing slash after 'no'. Let the original homepage, http://weekend.visitsweden.com/ also be redirected to http://weekend.visitsweden.com/no/ (the one with trailing slash) via 301.
So your redirection setup should be as follows:
http://weekend.visitsweden.com/ - via 301 to - http://weekend.visitsweden.com/no/
This should fix the issue. Essentially, we are eliminating the redirection loop here.
By the way, both the URLs, with and without trailing slash appear in Google when searched with the following queries:
Best regards,
Devanur Rafi
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl and Indexation Error - Googlebot can't/doesn't access specific folders on microsites
Hi, My first time posting here, I am just looking for some feedback on a indexation issue we have with a client and any feedback on possible next steps or items I may have overlooked. To give some background, our client operates a website for the core band and a also a number of microsites based on specific business units, so you have corewebsite.com along with bu1.corewebsite.com, bu2.corewebsite.com. The content structure isn't ideal, as each microsite follows a structure of bu1.corewebsite.com/bu1/home.aspx, bu2.corewebsite.com/bu2/home.aspx and so on. In addition to this each microsite has duplicate folders from the other microsites so bu1.corewebsite.com has indexable folders bu1.corewebsite.com/bu1/home.aspx but also bu1.corewebsite.com/bu2/home.aspx the same with bu2.corewebsite.com has bu2.corewebsite.com/bu2/home.aspx but also bu2.corewebsite.com/bu1/home.aspx. Therre are 5 different business units so you have this duplicate content scenario for all microsites. This situation is being addressed in the medium term development roadmap and will be rectified in the next iteration of the site but that is still a ways out. The issue
Intermediate & Advanced SEO | | ImpericMedia
About 6 weeks ago we noticed a drop off in search rankings for two of our microsites (bu1.corewebsite.com and bu2.corewebsite.com) over a period of 2-3 weeks pretty much all our terms dropped out of the rankings and search visibility dropped to essentially 0. I can see that pages from the websites are still indexed but oddly it is the duplicate content pages so (bu1.corewebsite.com/bu3/home.aspx or (bu1.corewebsite.com/bu4/home.aspx is still indexed, similiarly on the bu2.corewebsite microsite bu2.corewebsite.com/bu3/home.aspx and bu4.corewebsite.com/bu3/home.aspx are indexed but no pages from the BU1 or BU2 content directories seem to be indexed under their own microsites. Logging into webmaster tools I can see there is a "Google couldn't crawl your site because we were unable to access your site's robots.txt file." This was a bit odd as there was no robots.txt in the root directory but I got some weird results when I checked the BU1/BU2 microsites in technicalseo.com robots text tool. Also due to the fact that there is a redirect from bu1.corewebsite.com/ to bu1.corewebsite.com/bu4.aspx I thought maybe there could be something there so consequently we removed the redirect and added a basic robots to the root directory for both microsites. After this we saw a small pickup in site visibility, a few terms pop into our Moz campaign rankings but drop out again pretty quickly. Also the error message in GSC persisted. Steps taken so far after that In Google Search Console, I confirmed there are no manual actions against the microsites. Confirmed there is no instances of noindex on any of the pages for BU1/BU2 A number of the main links from the root domain to microsite BU1/BU2 have a rel="noopener noreferrer" attribute but we looked into this and found it has no impact on indexation Looking into this issue we saw some people had similar issues when using Cloudflare but our client doesn't use this service Using a response redirect header tool checker, we noticed a timeout when trying to mimic googlebot accessing the site Following on from point 5 we got a hold of a week of server logs from the client and I can see Googlebot successfully pinging the site and not getting 500 response codes from the server...but couldn't see any instance of it trying to index microsite BU1/BU2 content So it seems to me that the issue could be something server side but I'm at a bit of a loss of next steps to take. Any advice at all is much appreciated!0 -
E-Commerce Site Collection Pages Not Being Indexed
Hello Everyone, So this is not really my strong suit but I’m going to do my best to explain the full scope of the issue and really hope someone has any insight. We have an e-commerce client (can't really share the domain) that uses Shopify; they have a large number of products categorized by Collections. The issue is when we do a site:search of our Collection Pages (site:Domain.com/Collections/) they don’t seem to be indexed. Also, not sure if it’s relevant but we also recently did an over-hall of our design. Because we haven’t been able to identify the issue here’s everything we know/have done so far: Moz Crawl Check and the Collection Pages came up. Checked Organic Landing Page Analytics (source/medium: Google) and the pages are getting traffic. Submitted the pages to Google Search Console. The URLs are listed on the sitemap.xml but when we tried to submit the Collections sitemap.xml to Google Search Console 99 were submitted but nothing came back as being indexed (like our other pages and products). We tested the URL in GSC’s robots.txt tester and it came up as being “allowed” but just in case below is the language used in our robots:
Intermediate & Advanced SEO | | Ben-R
User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkout
Disallow: /9545580/checkouts
Disallow: /carts
Disallow: /account
Disallow: /collections/+
Disallow: /collections/%2B
Disallow: /collections/%2b
Disallow: /blogs/+
Disallow: /blogs/%2B
Disallow: /blogs/%2b
Disallow: /design_theme_id
Disallow: /preview_theme_id
Disallow: /preview_script_id
Disallow: /apple-app-site-association
Sitemap: https://domain.com/sitemap.xml A Google Cache:Search currently shows a collections/all page we have up that lists all of our products. Please let us know if there’s any other details we could provide that might help. Any insight or suggestions would be very much appreciated. Looking forward to hearing all of your thoughts! Thank you in advance. Best,0 -
Website Ranks and gets de indexed ??
Hi My website is almost 3-4 months old . Whats strange is that as soon as it get Crawled it ranks for few terms for 1-2 days and all of a sudden gets de Indexed for these same terms or Rank drops like drops from page 5 to page 10 . Nothing shows up in Webmater tools under Manual Action . Assuming its a Algorithmic penalty, How to deal with this kind of stuff. Should I stop working on this site all together ? Or assuming its a New website, google does not want it to rank for medium or high volume keywords ? What keywords I am after have 300 -2k searches per month .
Intermediate & Advanced SEO | | aus00070 -
Website Does not index in any page?
I created a website www.astrologersktantrik.com 4 days ago and fetch it with google but still my website does not index on google as the keywords I use is with low competition but still my website does not appear on any keywords?
Intermediate & Advanced SEO | | ramansaab0 -
Google and PDF indexing
It was recently brought to my attention that one of the PDFs on our site wasn't showing up when looking for a particular phrase within the document. The user was trying to search only within our site. Once I removed the site restriction - I noticed that there was another site using the exact same PDF. It appears Google is indexing that PDF but not ours. The name, title, and content are the same. Is there any way to get around this? I find it interesting as we use GSA and within GSA it shows up for the phrase. I have to imagine Google is saying that it already has the PDF and therefore is ignoring our PDF. Any tricks to get around this? BTW - both sites rightfully should have the PDF. One is a client site and they are allowed to host the PDFs created for them. However, I'd like Mathematica to also be listed. Query: no site restriction (notice: Teach for america comes up #1 and Mathematica is not listed). https://www.google.com/search?as_q=&as_epq=HSAC_final_rpt_9_2013.pdf&as_oq=&as_eq=&as_nlo=&as_nhi=&lr=&cr=&as_qdr=all&as_sitesearch=&as_occt=any&safe=images&tbs=&as_filetype=pdf&as_rights=&gws_rd=ssl#q=HSAC_final_rpt_9_2013.pdf+"Teach+charlotte"+filetype:pdf&as_qdr=all&filter=0 Query: site restriction (notice that it doesn't find the phrase and redirects to any of the words) https://www.google.com/search?as_q=&as_epq=HSAC_final_rpt_9_2013.pdf&as_oq=&as_eq=&as_nlo=&as_nhi=&lr=&cr=&as_qdr=all&as_sitesearch=&as_occt=any&safe=images&tbs=&as_filetype=pdf&as_rights=&gws_rd=ssl#as_qdr=all&q="Teach+charlotte"+site:www.mathematica-mpr.com+filetype:pdf
Intermediate & Advanced SEO | | jpfleiderer0 -
Pages Returning A 403 Error
Hiya Moz Community I hope you are all great, I have a question regarding one of my websites, I have the main site and 2 sub folder sites essentially, I decided to upgrade one of the sites and placed it in a different sub folder, I then set up a 301 redirect to the new location, so far so good, I have been having a look at my link profile using AHrefs, inside there is an SEO report facility, I ran the report and I have over 500 pages returning a 403 or Forbidden error. my question is whether the Equity from those pages is being passed to the new site? I actually removed all the old site from Google Cache to avoid misleading visitors, I suppose I could set the re-directs up manually if I the equity is not being passed to the new site although I was under the impression it would be, or 85% - 90% of it would be anyway. The reason why I am asking is that I have seen a significant drop in rankings for keywords that my site has always ranked highly for. thought I would see if you guys can clear that up for me. Thanks and regards Wes Dunn
Intermediate & Advanced SEO | | wesdunn19770 -
Webmaster Tools Not Indexing New Pages
Hi there Mozzers, Running into a small issue. After a homepage redesign (from a list of blog posts to a product page), it seems that blog posts are buried on the http://OrangeOctop.us/ site. The latest write-up on "how to beat real madrid in FIFA 15", http://orangeoctop.us/against-real-madrid-fifa-15/ , has yet to be indexed. It would normally take about a day naturally for pages to be indexed or instantly with a manual submission. I have gone into webmaster tools and manually submitted the page for crawls multiple times on multiple devices. Still not showing up in the search results. Can anybody advise?
Intermediate & Advanced SEO | | orangeoctop.us0 -
Google webmaster Smartphone errors fix
I have certain URL's that I have fixed before in google webmaster. With smartphone addition. It start appearing again. How can I fix the Google webmaster errrors for smartphones?
Intermediate & Advanced SEO | | csfarnsworth0