I wish I had an answer for how to stop the bots from hitting your site at all - I don't think a good one exists, as any solutions that wouldn't also block real human traffic to your site are going to be easy for spam bots to get around. I think your best bet is just to do everything you can to keep your data as clean as possible.
Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Posts made by RuthBurrReedy
-
RE: Help Blocking Crawlers. Huge Spike in "Direct Visits" with 96% Bounce Rate & Low Pages/Visit.
-
RE: Help Blocking Crawlers. Huge Spike in "Direct Visits" with 96% Bounce Rate & Low Pages/Visit.
Hi Eric,
A few things to reassure you off the bat:
- For what it's worth, there is a huge, HUGE amount of crawler spam happening in the web today. Every site I work on is being hit hard with false referrals and direct visits. I know Google Analytics is working on a solution to better filter these visits out. So I wouldn't be too concerned that it is something a competitor is doing to your site, specifically - it's more likely that it's been caught up in the general wave of spam crawlers.
- It's important to note that when we talk about Google looking at bounce rate and dwell time as part of ranking your site, those numbers are specifically from clicks through from search - that's data that Google can get without using your private web analytics data as a ranking factor, which they've said repeatedly that they don't and won't do. So a bunch of direct visits with high bounce rates will NOT affect your rankings.
So, it's not dangerous, just annoying. On to how to get that data out of your reports:
- Make sure you're not filtering out spam referrers at a View level - this can cause those visits to incorrectly appear as direct traffic.
- You could set up an Advanced Segment in Google Analytics to filter out direct visits with visit times of, say, under 5 seconds. Some real traffic may get caught in that, but it will get the noise levels down.
- The best way to filter out spam bot traffic, in my opinion, is to set up hostname filtering. Here's a post on Megalytic on how to do that: https://megalytic.com/blog/how-to-filter-out-fake-referrals-and-other-google-analytics-spam. Make sure you've also got an "Unfiltered Data" View so you'll still have historic raw data if you need it.
Hope that helps! Good luck.
-
RE: Internal Linking: What is the best practice for pages not included in Nav bar?
You certainly don't need to include every page of your website in your top navigation menu. Your plan of having a Locations page that then links to each of your location pages individually is a fine way to go.
That said, the deeper into your site architecture your page is, the fewer ways there are for people and search engines alike to discover it - to your point, there is now only one page on your site linking to all of these location pages. One reason internal linking outside the navigation is important is that it provides additional ways for users and search engines to browse to your content. I would recommend taking a look at the pages on your site and thinking about what pages a user might want to visit next, and linking to those. Providing an intuitive next step for your users keeps them engaged, and provides additional ways for your content to get discovered.
-
RE: How to check if an individual page is indexed by Google?
This is what I would do, too: search for a large chunk of text from the page and see if the page comes up. Site: is not always 100% accurate.
-
RE: I have a GoDaddy website and have multiple homepages
You are encountering something that happens from time to time with communication about websites - as far as GoDaddy is concerned, there is only one "page" - there is only one document that is your home page - but the page has several different URLs, which is what's causing Google and Moz Tools to recognize it as several different pages. It's more of a terminology thing than anything (but a sarcastic response is never in good taste, not the best customer service there)!
As far as website content management systems go, I always recommend WordPress. It's very customizable, but there are many great out-of-the-box templates that work well for SEO too. There are several great plugins for WordPress that help make SEO changes easier - two good ones are WordPress SEO by Yoast and the All-in-One SEO pack. If you want to change your website's hosting as well, do a bit of research - many hosting companies will help you with step-by-step instructions for moving your site and installing WordPress.
-
RE: I have a GoDaddy website and have multiple homepages
Jaume is correct that the best solution would be to 1.) change that 302 redirect to a 301 (which would let Google know that the redirect is permanent, rather than temporary) and 2.) Also use a 301 redirect to redirect http://www.ecuadorvisapros.com/home.html to http://www.ecuadorvisapros.com. http://www.edcuadorvisapros.com is the home page that you want. However, this can be difficult to do if you don't have developer experience, and how easy it is to do also depends on the website system you're using. Here's an article on the godaddy.com site about how to do it: https://www.godaddy.com/help/using-301-page-redirects-234
If you don't want to mess with redirects, or can't get the page redirecting properly, you will want to use the canonical tag instead. In the section of http://www.ecuadorvisapros.com/home.html, put the tag . This will let Google know which version of the page is the "official" version. You can learn more about the canonical tag in the Moz learn section, here: https://moz.com/learn/seo/canonicalization.
-
RE: Removing duplicate content
Are you basing this on a site: search? It's fairly common for URLs to appear in a site: search that otherwise will not appear for any actual searches. Are the undesirable versions of the URLs getting any search traffic?
-
RE: Javascript onclick redirects / porn sites...
Hi Marcy,
If the sites are using your brand name and/or other brand terms, and your brand is copyrighted, you may be able to file a Digital Millennium Copyright Act takedown request with Google: https://www.google.com/webmasters/tools/dmca-notice?pli=1. As they note in the description on the tool, be very clear about whether the other site's actions actually constitute a violation of your copyright before filing the request.
I think it's unlikely that these new sites are impacting your site's performance in search - I was a little unclear about the JavaScript redirect, though (I'm at work and don't want to click on the links you posted on my work computer). Is it redirecting from their site to your site, or from their site to another site that is the porn/junk site? If it's the latter, that shouldn't be affecting your site at all. If it's the former, you may want to file disavow requests at the domain level for those sites just in case.
If your drop in rankings was caused by these new sites, I would expect to see a drop in performance across the board, rather than for specific queries, so I recommend that you keep digging on other reasons for the drop. I would take a look at the sites that are ranking now for the terms you've lost rankings for. How are they different from your site? What sites are ranking now that weren't ranking when you were on top? It may be that Google has decided that your site doesn't fulfill the search intent for those keywords, so taking a look at the sites that rank now will give you some insight into the kinds of pages that Google wants to rank for these terms. Since these were highly-converting terms for you, consider investing in PPC ads for these terms while you work to regain your organic presence. Good luck!
-
RE: Help! The website ranks fine but one of my web pages simply won't rank on Google!!!
Don't forget that every keyword is different - how you rank depends on what you're doing compared to other sites targeting that term, not just what you're doing on your own site. So some keywords just take a larger, higher-authority link profile to rank for than others. A good place to start with getting links for that page would be to look at the backlinks that other pages that rank for that term have - you may be able to get some links from the same or similar sites.
-
RE: Help! The website ranks fine but one of my web pages simply won't rank on Google!!!
Ah OK, thanks for the clarification!
That problem, to me, sounds like you need some links! In general when Google is ranking your home page for a term, instead of the page that is actually about that term, it's because they recognize that your site has some topical relevance for that term, but the individual page doesn't seem that important based on how many links are pointing to it. Are there ways you can flow some additional internal link juice to that page? Are there sites that are linking to your home page right now that are very closely related to the topic of the page in question, that you could ask to point to that page instead? Are there topically-related sites that don't link to that page right now that you could possibly get a link from? All of these will beef up your page authority, which should help.
In terms of your copy being too far down on the page - if you don't think it will negatively impact your user experience, you could try moving it up, or integrating it into the section with the images, but I don't know how much that will help. You also may need more copy on the page - if your page is 300 lines of code long, and only 5 of those lines are unique copy, it's hard to send a strong enough signal of relevance. Can you expand what you say on the page to make it a better resource on the topic at hand?
-
RE: Help! The website ranks fine but one of my web pages simply won't rank on Google!!!
Is the problem that the page isn't appearing in the index, or that it isn't ranking for its target terms?
If the page has a lot of images but doesn't otherwise have much copy, it may be that Google is determining it to be too similar to other pages on your site and so is not displaying it. If it's not being indexed at all (doesn't show up in a site: search or when you search for a block of copy in quotations), double-check that your robots.txt isn't blocking it and that you don't have a meta robots noindex tag on the page. The suggestion of running Screaming Frog on your site to make sure a crawler can find the page is a good one - Screaming Frog will also tell you if the page is returning a weird HTTP status or is blocked by robots.