Direct traffic is up 2100% (due to a bot/crawler I believe)
-
Hi,
The direct traffic to website www.webgain.dk has increased by over 2100% recently. I can see that most of it is from US (my target audience is in Denmark and the website is in danish).
What can I do about this? All this traffic gives my website a bounce rate of 99.91% for direct traffic. I believe it is some sort of bot/crawler. -
Already done. They also included the tip in their newsletter for beta-testers.
-
You might want to let them know about this, so they can add in documentation so future users know what is up before panicking.
-
Follow up: I have fixed this now. It was a monitoring tool by Digicure, where I have signed up to be a beta tester. Their platform checks the website like a normal visitor from servers around the world (in my test case it is Denmark and California) and thereby it looked like normal direct traffic in my data. I excluded their stated server IP-addresses in my Google Analytics filters and that helped. Thanks again guys for the help.
-
Thank you for all your great advice. I will follow them and see how it works.
-
If you are running Wordpress also check what page / pages are being accessed. I have had bots nail my wp-login like that before. If that is the case harden your installation, one thing I have found that stopped it was setting a deny in the htaccess on wp-login / wp-admin.
-
I was having the same problem ( for me it seemed to be Bings ads bot) . I used this guide below and it seems to filter out most of the bot visits.
-
I would check the service providers first just to know for sure they're all coming from the same provider. You can check this by visiting your Audience > Technology > Network report on the left side of your Google Analytics. If you see the same network and browsers being used I would use a filter (only if you're really determined/ 100% sure that it's bot traffic) to get them completely out of your Google Analytics view.
-
It's weird that the bot is accepting cookies, but with a bounce rate that high, I agree it's probably something automated (though it could be people who were looking for something else or were directed there by an email or an app accidentally). You can look through your logs to see IP addresses and then do as Atakala says and block the traffic if you're worried about bandwidth. You can also just filter it out in GA by excluding US traffic (if your'e worried about analytics being messed up).
-
It's probably AWS, which is amazon buyable crawl service.
If ıt costs you too much then try to ban it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Multiregional / Multilingual SEO - What do you do when there is no equivalent page?
Hello, We're building out a small number of pages for the US in a sub-folder .com/us. The idea is to show US specific pages to users in that location. However, we also have a number of pages which we will not be creating for the US as they're not relevant. I am planning on geo-targeting the US folder to instruct the search engines that this subfolder should appear in the US SERPS but since it isn't an exact science, there is a chance that US visitors may land on these non-us pages which could potentially give them a bad user experience. What should we do in instances where a US user lands on a non-us page with no equivalent page? Any help would be much appreciated!
International SEO | | SEOCT1 -
Multiregional / Multilingual SEO - Subfolders Question
Hello all, I wonder if you can help me... I have a question about subfolders in multi-regional / multi-lingual SEO - more specifically in reference to targeting the UK and the US. Having looked at some global websites these are the types of implementations I've most commonly seen: UK subfolders .com/uk .com/gb .com/gb/en-gb | .com/en-GB .com/gb-en .com/en-gb .com/uk/en US subfolders .com/us .com/us/en-us | .com/en-US .com/us-en .com/en-us .com/us/en Are any of these approaches better than others or is it all a matter of personal preference? What's the reason for using .com/gb over .com/uk (or vice versa) for example? Secondly, my assumption is that the examples above which include language subfolders do so because these companies are targeting different speaking users within these countries. Would I be right to think that since the organisation I work for is only targeting the American speakers in the US, we wouldn't need to go so far as to have language subfolders in addition to location subfolders? Would be great to get some feedback / suggestions! Thanks!
International SEO | | SEOCT0 -
Why Google is not indexing each country/language subfolder on the ranks?
Hi folks, We use Magento 2 for the multi-country shops (its a multistore). The URL: www.avarcas.com The first days Google indexed the proper url in each country: avarcas.com/uk avarcas.com/de ... Some days later, all the countries are just indexing / (the root). I correctly set the subfolders in Webmaster tools. What's happening? Thanks
International SEO | | administratorwibee0 -
International Confusion between .com and .com/us
A question regarding International SEO: We are seeing cases for many sites that meet these criteria: -International sites that have www.example.com/ ip redirecting to country site based on ip redirect (ex. www.example.com/ 301 to www.example.com/us -There is a desktop + mobile site (www.example.com + m.example.com) The issue we see is Google shows www.example.com/ in US search results instead of www.example.com/us in search results. Since the .com/ redirects, there is no mobile version, and www.example.com/ also shows up in mobile SERPs instead of m.example.com/us. My questions are: 1. If www.example.com/ is redirecting users and Googlebot, why is Googlebot caching it with the content of www.example.com/us? 2. Why is www.example.com/ showing up in SERPs instead of www.example.com/us? 3. How can we help Google display www.example.com/us and m.example.com/us in SERPs instead of www.example.com/? Thanks!!
International SEO | | FranFerrara0 -
International SEO question domain.com vs domain.com/us/ , domain.com/uk etc.
Hi Mozzers, I am expanding a website internationally. I own the .com for the domain. I need to accommodate multiple countries and I'm not sure if I should build a folder for /us/ for United States or just have the root domain .com OPTION 1:
International SEO | | jeremycabral
domain.com/page-url -- United States
domain.com/de/page-url -- Denmark
domain.com/jp/page-url -- Japan OPTION 2:
domain.com/us/page-url -- United States
domain.com/de/page-url -- Denmark
domain.com/jp/page-url -- Japan My concern with option 2 is there will be some dilution and we wouldn't get the full benefit of inbound links compared to Option 1 as we would have geo ip redirection in place to redirect users etc. to the relative sub-folder. Which option is better from an SEO perspective? Cheers, Jeremy0 -
If domain mapping subfolders to TLD's is it perceived as a fully separate entity/site therafter ?
Hi I take it once you have domain mapped a country specific subfolder to a country specific TLD (for better local region targeting reasons) Google perceives it as a completely separate entity and it no longer shares any of the parent sites domain benefits (such as domain authority etc) so from that point on requires its own dedicated link building etc ? All Best Dan
International SEO | | Dan-Lawrence0 -
SEO for Subdomains for different languages .com/fr, .com/es
Hi All, I was wondering how best to to approach optimisation of a site that exists on a single .com domain, but has different subfolders for different languages. The site is a .com and it has subfolders for French, Spanish, Russian and English. The business is situated in France and the vast majority of clients are French and English speakers. I've read that it's possible to geo target these subfolders using webmaster tools however I believe this is an inferior method of optimisation than having tld's. Just wondered if anyone had experience of htis and could provide any advice ? As they won't be rebuilding the site for another year or so I wondered if there were any quick wins? My second question is to do with how best to set these campaigns up within SEO Moz. would it be better to track at a subdomain or subfolder leverl (for different languages)? If someone could advise I would greatly appreciate it! Thanks, vantresca
International SEO | | vanvallejo0 -
Multi Language / target market site
What is the best way to deal with multiple languages and multiple target markets? Is it better to use directories or sub-domains: English.domain.com Portuguese.domain.com Or Domain.com Domain.com/Portuguese Also should I use language meta tags to help the different language versions rank in different geographic areas e.g. Are there any examples of where this has been done well?
International SEO | | RodneyRiley0