How is my competition causing bad crawl errors and links on my site
-
We have a compeditor who we are in a legal dispute at the moment, and they are using under hand tactics to cause us to have bad links and crawl errors and i do not know how they are doing it or how to stop it.
The crawl errors we are getting is the site having two urls together, for example www.testsite.com/www.testsite.com and other errors are pages that we do not even have or pages that are spelt wrong or have a dot after the page name.
We have been told off a number of people in our field that this has also happened to them and i would like to know how they are doing it so we can have this stopped
Since they have been doing this our traffic has gone down by half
-
Hi no there is only me who deals with the site. I have put copyright notice on the site but i will read about authorship accorss the site.
-
Hi Diane,
I like the way Ryan thinks!
...but I am hoping we won't have to go to the length of having to resort to a content bomb.The lesson in a situation like this is to realize that being a good white hat SEO unfortunately means needing an understanding of some of the tactics used by black hats
or maybe just a little help from some friends Since there is obviously an issue with your content being copied, the first thing I would do is to implement Authorship markup across your entire site. By doing this you ensure that any content that is "borrowed" is immediately "outed" to the search engines because it doesn't have an external link from your Google profile page, which acts as verification that you are in fact the author of the content. Matt Cutts and Othar Hansson have given a really easy rundown on implementation in this Google Webmaster Help video
For the moment though, it would be better if you can avoid making any changes to the site until we can identify all of the issues in play.
BTW ... I think I have an inkling of what might be going on here ... is there a programmer or designer who is or has been involved in the development of the site besides yourself?
In the meantime, I'm continuing with a diagnostic based on the information we have and will let you know as soon as I have confirmed my suspicions or otherwise.
Hang in there,
Sha
-
May I suggest planting a "bomb" in your content?
Most thieves are lazy. Rather then create content themselves they steal from others. Their laziness is dependable.
Take a look at their copies and determine what content is and is not being stolen. If they are copying everything including the HTML code and meta tags, you can add canonical tags to your site and other helpful code.
If they are not copying the meta tags, you can add fake content and use the noindex, nofollow tag to protect your site, but provide content which otherwise would cause a site to be removed from Google's index. When they steal the content, it wont have the noindex tag and the site would get nailed.
There are many other possibilities but I am confident you can outsmart them if you try. In addition, be sure to report the site(s) to Google: http://www.google.com/support/bin/static.py?page=ts.cs&ts=1114905
Another idea is to copyright your work, thereby protecting it and giving you legal proof the content is yours.
-
thanks for this, will send private message. we have had to redo the site so many times with new content because the content keeps on being stolen by a franchise group who then pass it on to their franchisees.
-
Hi Diane,
Ryan's response is spot on and his suggestions are excellent.
If you can provide the URL(s), then we can take a look and see exactly what is going on with the referring page(s).
If you don't want to share the information publicly in the Q&A, you can private message each of us through your SEOmoz profile page.
If you ever think that someone has access to edit pages on your site without your permission, the first thing to do is to check with your service provider whether there are any active ftp accounts that you are unaware of. I have seen situations before where people have managed to get a "back door" set up and then it is as simple as logging in and changing pages without your knowledge.
Given that this involves a legal dispute, if we can do a proper diagnosis and trace the source of the errors (or if there happens to be a back door in place), then you would be able to:
- issue a cease and desist
- better secure the server against unauthorized access
- Ban any ip addresses identified as malicious
Hope that helps,
Sha
-
You mentioned these are crawl errors. Are you using the SEOmoz crawl report? If so, please look at the "referrer" field. It will offer the page on your site which is providing the bad URL.
If you are willing to share the referring page, we can take a look and possibly provide more detail.
Anyone can create bad links to your site which can appear in Google or Bing WMT. Only someone with the ability to add content on your site can create crawl errors. Either your site is open to user generated content and someone created a bad link, or someone with access to your web server created the content.
-
will do thanks
-
If iit is happniong to many, then maybe its a reason not to suspect them, but like i say go to Bing WMT and have a look at where the links ae comming from.
-
the reason why i know they are behind it, is because other companies in the field of the website have had the same problems and after doing research for our legal team over this matter, we spent time speaking to over 40 people in the field of the website and found it happened to them aswell.
These people are cowboys but hopefully it will all be sorted out soon.
-
I would look in Bing WMT to see where the links are comming from for a start.
i must say also, If you dont know how they are doing it, then maybe you dont know if they are doing it., it may be somthing quite inocent.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moving from no follow to follow links on our eCommerce site
Hi everyone, I recently taken on an SEO eCommerce account and found that all the footer links have a no follow attribute. I've requested that the no follow tags be removed as the pages are quite valuable (about us, finance, recycling, help centre etc). I've been asked what the risks are and all I can think of is a slightly increased number of pages for Google to Crawl. Are there any other risks you can think of? Does anyone have experience around making this type of change? For benefits, I believe that it will make our content look more trustworthy to Google and help with traffic through to those pages in the SERPs. Any other pros you can think of will be a great help.
Technical SEO | | RebekahVP0 -
My site doesnt give any 404 error
Hi guyz, I've realized that when someone try to access some url that doesn't exist on my site, my site gives a custom 404 page but not give any 404 http status code.
Technical SEO | | atakala
It still give 200 http status code. My system is IIS based how can I solve it?0 -
Paid links that are passing link equity from a blog?
We have a well-known blogger in our industry with whom we've had a long-standing relationship. We've had inbound links from his blog for many, many years. Today I noticed that we are running a banner ad listed on all pages of his blog under a heading that says "Sponsors." He has dedicated an entire page of his site giving full disclosure of all advertising. However, all of the links on his site pointing to us are passing link equity. To my knowledge they've been this way ever since they were first established years ago. I am fairly certain this fellow, with whom we have an excellent relationship, neither knows nor cares what a "nofollow" attribute is. I am afraid that if I contact him with a request that he add "nofollow" attributes to all of our links that it will damage our relationship by creating friction. To someone who knows nothing and cares nothing about SEO, asking them to put a "nofollow" on a link could either seem like a technical request they don't know how to handle, or something even potentially "shady" on our part. My question is this: Considering how long these links have been there, is this even worth worrying about? Should I just forget about it and move on to bigger fish, or, is this a potentially serious enough violation of Google Webmaster guidelines that we should pursue getting those links "nofollow" attributes added? I should add that we haven't received any "unnatural" link notifications from Google, ever, and haven't ever engaged in any questionable link-building tactics.
Technical SEO | | danatanseo1 -
Site links show spam
Hi folks, I'm working on a website that runs on WordPress and was not updated by the owner, this has resulted in a malware injection and now when you search the companies name in Google, the site links appear with words like Viagra, et al. I've seen this a number of times, so I went through the code and have removed all the malware. I presume I now have to wait for Google to recrawl the website and update the site links? Is there anything else I should be doing to speed up the process? Thank you 🙂
Technical SEO | | ChristopherM0 -
Site maintenance and crawling
Hey all, Rarely, but sometimes we require to take down our site for server maintenance, upgrades or various other system/network reasons. More often than not these downtimes are avoidable and we can redirect or eliminate the client side downtime. We have a 'down for maintenance - be back soon' page that is client facing. ANd outages are often no more than an hour tops. My question is, if the site is crawled by Bing/Google at the time of site being down, what is the best way of ensuring the indexed links are not refreshed with this maintenance content? (ie: this is what the pages look like now, so this is what the SE will index). I was thinking that add a no crawl to the robots.txt for the period of downtime and remove it once back up, but will this potentially affect results as well?
Technical SEO | | Daylan1 -
How do you find bad links to your site?
My website has around 900 incoming links and I have a Google 50 penalty that is sitewide. I have been doing research and from what I can see is that the 50 penalty is usually associated with scetchy links. The penalty started last year. I had about 40 related domains to my main site and each had a simple one page site with a link to the main site. (I know I screwed up) I cleaned up all of those links by removing them. The single page site still exist, but they have no links and several of them still rank very well. I also had an outside SEO person that bought a few links. I came clean with Google and told them everything. I gave them all of my sites and that the SEO person had bought links. I gave them full disclosure and removed everything. I have one site that I can't get the link removed from. I have contacted them numerous times to remove the link and I get no response. I am curious if anyone has had a simular experience and how they corrected the situation. Another issue is that my site is "thin" because its an ecommerce affiliate site and full of affiliate links. I work in the costume market. I'm also afraid that I have other bad links pointing to my site. Dooes anyone know of a tool to identify bad links that Google may be penalizing me for at this time. Here is Google's latest denial of my reconsideration request. Dear site owner or webmaster of XXXXXXXXX.com. We received a request from a site owner to reconsider XXXXXXXX.com for compliance with Google's Webmaster Guidelines. We've reviewed your site and we believe that some or all of your pages still violate our quality guidelines. In order to preserve the quality of our search engine, pages from XXXXXXXXXX.com may not appear or may not rank as highly in Google's search results, or may otherwise be considered to be less trustworthy than sites which follow the quality guidelines. If you wish to be reconsidered again, please correct or remove all pages that are outside our quality guidelines. When such changes have been made, please visit https://www.google.com/webmasters/tools/reconsideration?hl=en and resubmit your site for reconsideration. If you have additional questions about how to resolve this issue, please see our Webmaster Help Forum for support. Sincerely, Google Search Quality
Technical SEO | | tadden0 -
Are lots of links from an external site to non-existant pages on my site harmful?
Google Webmaster Tools is reporting a heck of a lot of 404s which are due to an external site linking incorrectly to my site. The site itself has scraped content from elsewhere and has created 100's of malformed URLs. Since it unlikely I will have any joy having these linked removed by the creator of the site, I'd like to know how much damage this could be doing, and if so, is there is anything I can do to minimise the impact? Thanks!
Technical SEO | | Nobody15569050351140 -
Minisites - 301 Redirect or Links to Main site
Not sure whether this is considered black hat or not but I know it is done and I would like to know which is the most effectrive method. If you were to acquire multiple sites in the same niche to your main site (either by buying existing sites or perhaps registering expired domains) which already had strong aged backlinks, is it better to either: a) 301 the new domain to the main site (or a subpage perhaps) b) create 'minisites' on the new domains (trying to mirror the URL structure of the previous incarnation if possible to scoop up and remaining inbound backlink juice, on seperate IPs to the main site as well) and then place several links to the main site & subpages. Would the decay of link juice through 301's mean you lose benefit that way or is it the same as a normal link? Would the 301 method mean any IBL's into URL's other than the homepage be lost? The homepage of the minisite will likely have 4 or 5 internal links so will this dilure the effect of the links to the main site? Thanks in advance!
Technical SEO | | OzDave0