Bad Domain Links - Penguin? - Moz vs. Search Console Stats?
-
I've been trying to figure out why my site www.stephita.com has lost it's google ranking the past few years. I had originally thought it was due to the Panda updates, but now I'm concerned it might be because of the Penguin update. Hard for me to pinpoint, as I haven't been actively looking at my traffic stats the past years.
-
So here's what I just noticed. On my Google Search Console - Links to your Site, I discovered there are 301 domains, where over 75% seem to be spammy. I didn't actively create those links.
-
I'm using the MOZ - Open site Explorer tool to audit my site, and I noticed there is a smaller set of LINKING DOMAINS, at about 70 right now. Is there a reason, why MOZ wouldn't necessarily find all 300 domains?
-
What's the BEST way to clean this up??? I saw there's a DISAVOW option in the Google Search Console, but it states it's not the best way, as I should be contacting the webmasters of all the domains, which is I assume impossible to get a real person on the other end to REMOVE these link references.
HELP!
What should I do? -
-
It's better to 301 a page if it's relevant than it is to let it go 404. However if it's a service or product you no longer carry, might be better off keeping it a 404 so the page goes away unless you think you'll be using that url again.
404s won't hurt you unless you have a huge number of them, but if you have a page you no longer want and wish for it to be forgotten about, you can let it 404 and will eventually go bye bye.
-
I don't know how that .js subpage is looping in the address bar. I'm just using the foundation.zurb.com code to develop my site as mobile friendly.
I do have an idea why the articles page might be spinning out of control. It's a dynamic page that pulls my articles content, and it's dependent on the URL, so technically I could have an infinite number of pages spawning. I will correct the script to realize this error, and have it redirect to a base page.
As for the other previous cloaking pages, I'm assuming if I do a 301 redirect to the homepage, that would be better than to DELETE the page entirely which would return a 404.
-
I ran your site url through Screaming Frog it found more pages than I expected at 30% and was still counting so I took a quick glance at the urls it was returning and found hundreds of urls containing the same words like " vendor " , " foundation " , " js " over and over and over again. Then just did a search for " .js subpage looping ".
About moz saying what your page count was, this is what threw me off, since after I saw the ungodly high amount of urls I did a site:yourdomain.com search in google to see how many pages were being indexed and was returned a normal amount somewhere around 1,240 pages.
The cloaking / anything blackhat is a big deal to get off as soon as possible, even the left over script could be seen as an issue. But in this case it's also causing extra pages to be seen so removing that script will help clear up a great bit of google's confusion, since if screaming frog was having issues so was google then, and could even be why you dropped in ranking.
Screaming frog saw these pages as various different ones:
All from the article subpage, so I guess that's the home of that script or at least where it starts to make new pages.
-
Thank for helping out with the site audit! I truly appreciate it.
How did you detect the loop in the link structure?
Regarding the page count, the current count I thought was < 1000 based on MOZ Site Crawl tool.
** Prior to January of this year (2016), I had a cloaking page that was running that basically generated pages with a different city reference. So my site came up for searches like "Wedding invitations Banff, W I Timbuktuu, W I any city". Obviously this was a black-hat tactic, which I figured out a few months ago is a NO-NO with the PANDA update. So I unreferenced the link from my link structure, and I also requested that the directory structure be removed from the Google Webmaster Tools. I just realized I didn't take that script offline, or have it re-direct - i.e. www.stephita.com/wedding/type_anything_here is still live. But there is no LINK from my site that would go there anymore. ** I'm thinking I should probably do a 301 redirect to my homepage instead now.
So I'm curious as to how Screaming Frog is seeing these pages, if it is indeed these pages. www.stephita.com/wedding/____________________
Thanks!
-
***Seems something is going on with your internal link structure, many many many weird urls with duplicate subpages:
Seems it's a .js subpage loop going on, http://wordpress.stackexchange.com/questions/93844/child-pages-loop
So that handles the huge page total issue, get this handled and maybe peek into other pages and see how they are worded and structured, make sure every page has it's h1 tag but only one h1, h2 - h5 for anything else if it needs it. For a 10 position move, I wouldn't imagine there is a huge number of issues going on. It's really a matter of finding them.
End Update
Curious as I went to run screaming frog on your site and its saying it's only 30% done crawling your site at 8,549 pages, then is showing me 33,359 pages and counting to go!
Do you really have this many pages?
If so, do you need them all?
If you have 30k + pages, odds are you might have some bad pages from all that. Having this amount of pages makes it very hard to do good SEO unless you've been doing good SEO since day one.
I'd say it's not a penalty since you only went from page 1 to page 2, that happens even on good days, often is a sign of competitors one uping you somehow.
If you can, I'd suggest pruning a good portion of these pages unless you have to have them, like they are product pages or such.
At the time of writing this, it's still totaling your total pages it's up to 53,932 pages now and still counting, screaming frog is at 29% to go from 31%.
-
Oh.. I was hoping it was a 'penalty' b/c at least there is hope that fixing those issues would make Google smile at my site again
I've basically started this 'clean-up' process the past 3 months, where I've used the MOZ tools to identify "high/medium priorities".I've put in the effort to remove all "duplicate pages" which were marked as HIGH, and I've adjusted the META TITLES which were marked as MEDIUM. According to MOZ crawl report, I'm down to a few issues vs. 900 or so when I first ran the report a few months ago.
As for the Penguin fixes, I only discovered this 3 days ago, and submitted my first DISAVOW list yesterday which contained 900+ urls and about 210 domains.
** If the 10 placement drop wasn't a penalty, I can only hope this 'clean-up' attempt can only help? But the big question is timeframe...
Is it possible Google would have only assessed my site a MINOR penalty? Or is it either nothing or SEVERE? -
As for the article about penguin being part of the core, I have read a few articles back in January of this year about that but nothing instead it was an unnamed update and then the adwords update. So far no one has 100% declared that penguin was a part of the core update yet.
And from what you've mention a 10 placement drop doesn't seem like a algo penalty, more like some tactics that were working on your site stopped, or other sites have updated their content and appear more relevant, causing your site to drop to page 2.
Let me give an example as to what penguin penalty could be like, I have a keyword for a client that would not rank passed page 4, and has grown to stay on page 6 and 7. After a recent site redesign where I made SEO top priority when I launched the site, about 2 weeks or so after we saw that keyword rise, from page 7 up to page 3. Stayed on page 3 for a day or two, then dropped back to page 6 then to page 7, stopping right back where it was. This most likely is a penguin penalty, especially after seeing a not so awesome backlink profile with excessive anchor text for that keyword to directories galore as well as comment spam.
I think if you were hit by an algo penalty that you'd be suffering ranking issues on a much more severe scale. I don't think your site sucks, it's just that you always have to keep your site up to date, gone are the days of posting content and walking away. You need to creating new content, promote it correctly, improve it, constantly check backlinks, check competitors, stay on top of current trends of your industry. And changes you make today, won't really show results in SERPs as fast as you'd like. Sometimes it can take time.
-
You are soooo right about the scouring the internet till I'm cross-eyed!
hahaahI read this article: http://www.brafton.com/news/seo-1/penguin-to-join-googles-core-algorithm-what-you-need-to-know/ That 'expert' says Penguin has already been put into the core algorithm. ** I can't seem to find anything that "Google" has said, do you know?
Here's a quick rundown, my site www.stephita.com, used to rank on near the top of page 1 for google.ca on search term "wedding invitations", some time ago (I haven't been keeping track, but I believe maybe 1-2 years ago probably) my site has dropped to page middle of page 2. So I'm thinking my site got dropped about 10 spaces. If I was hit with any penalties b/c of Panda & Penguin, does a 10 place drop seem reasonable, or would it have been more? A few facts:
-
I never got a manual action
-
I believe Panda penalty b/c I had some 'black-hat seo' pages, like a cloaking page, and also some relatively THIN content.
-
I believe Penguin b/c I just did a backlink audit, and felt I had about 200 spammy domains linking to me which amounted to about 900 urls.
So with those 3 points, does 10 place drop seem reasonable, or does my site just suck
hahaha -
-
Panda is part of the google core algorithm now, penguin is not yet, but is expected soon ( sometime between now and the end of this year soon )
So since penguin isn't yet part of the core algorithm, the key is to submit your disavow list as soon as possible, since it can take anywhere from a month to three months for google to nofollow those links you sent. They start nofollowing the urls once you send it in, but for reasons I have yet to gather aren't as speedy with this process as it is when you use the google fetch feature.
My understanding of how site' recover now from penalties are two fold.
If it's a manual penalty ( such as they get a message from google in their webmaster console ) you fix what they mention and submit a reconsideration request.
If it's a algorithmic penalty, you scour the internet and read as much as you can until you get a little crosseyed then you read some more, fix the issues that relate to your penalty ( or what you suspect is your penalty ) and then you wait, you tackle other aspects of your site's SEO and wait until that specific algorithm is rerun / updated and run, then it's not even 100% that your site will recover right away, might take a bit of time after that.
So once you do recover, stress to your client, your boss, or yourself that this cannot ever happen again, and to never ever ever neglect your site's SEO again or this issue will only be 100 times worse.
Hope this helps!
-
Thanks for the input!
What's your understanding of how sites recover from penalties now? I've seen a posting about how Panda and Penguin are now 'baked in' the search algorithm. Where as in the past, sites would have to wait till Google rolled out a Panda/Penguin update for sites to show any 'recovery'. So, if it is 'baked in', if I submitted my 'disavow' list a few hours ago, should I hopefully see immediate results now?
Thanks!
-
If those links aren't doing anything else but redirecting, yes, get rid of those, they aren't helping you and could hurt you if they haven't already.
If they were all from the same domain, i'd disavow that domain as well, but that doesn't seem to be the case. As it was mentioned above by another poster.
You shouldn't need to disavow eventwire.com unless there is a backlink from them to your site, and it's not helping anyone. But remember not to get carried away when using the disavow tool, just remember this nifty saying **" when in doubt, leave it out. when it stinks, disavow the links " .
** -
No one at google reads these, it's all done automatically now, the comments are for your usage when you look at the disavow list at a later date.
Yes, when you submit your next disavow file, say you disavowed 10 the 1st time and wanted to disavow 10 more, so your 2nd disavow file will have 20 urls in it, 10 from the last time and 10 from this time.
It's best course to download the text file from before and add the urls you want to remove this time to that txt file, add a comment about that time to help you remember or inform others.
Here is an article from 2014 from moz.com that helps explain this further - https://moz.com/blog/guide-to-googles-disavow-tool
Remember you don't have to comment these, but at the very least, comment each time you use the disavow feature at least, this will make things easier on you.
Just be careful, removing even spammy urls could alter your domain authority or possibly worse. always be 100% sure if you want the urls selected to be ignored, this process can take as little as a month and as much as 3 months with some people taking a year to have a url disavowed. And if you make a mistake, that's about how long it can take to correct it.
-
-
When submitting a disavow list, should I put a COMMENT on each one? Does someone at Google read it?
-
If I send 10 at a time, when I send the second batch, should I re-include the first batch with it? So basically just upload the same file over and over with new additions.
-
-
Just a " heads Up " the Original Poster was saying he had " 301 domains linking to his site, not that he has a domain 301ing to his site "
I've just come from this area for a recent client, Luckily for me it was only 700 urls total.
**I got all the links linking back to my site ( backlinks ) from: **
- Google Webmaster Tools ( Search console )
- Moz.com
- Ahrefs.com
- SEMrush
Added them all to a spreadsheet, downloaded an addon in google sheets called "remove duplicates" to remove same urls, once that was done, I put as much information for these urls as I could find, both what was already listed from those sites I got the backlinks from. Then I also went and visited every single one of them, listed if they were still working or 404, what the anchor text was, what the domain authority and page authority was and anything else to help, like was it a foreign site or a directory listing or comment spam.
I had over 75% of my backlink profile was nofollow or was a directory / comment spam.
So far I have disavowed 153 urls, I have more but want to do chunks at a time so if something really horrid and unforeseen happens I can reavow ( yes thats a real thing ) the urls a section at a time a wait.
I recommend doing this once to create a master backlink list, and just keep things dated and correct so you only have to add to it each month and keep up to date on your backlink profile!
Since backlinks are on par with content in terms of ranking there is no reason to not be so involved with backlinks.
-
Hi Todd,
Thanks again.
-
In your opinion, I should disavow those low quality directory listings? Would Google/Bing consider those as SPAM?
-
I thought those "stat" type referring links look more spammy then directory listings. Would these have "hurt" my ranking at all?
-
I've looked at my Google Analytic profile, and have noticed a steep decline the past 2 years. The fact that some of these links might have been there for a while, does cleaning it up 'now' greatly improve my standing?
So the drop in my SERP is based on users doing a keyword search for the term "wedding invitations", on google.ca. I started this company about 14 years ago, and pretty sure my domain has been on page 1 for the first 10-12 years. Especially in my local market in Toronto, we were top 3 results. I've only noticed earlier this year, that my site has dropped to page 2! (it probably dropped to page 2 much earlier, which probably coincides with my traffic drop, so let's say it has been on page 2 for the past 2 years). There are now 4 other local based companies that sit ahead of my site, along with some large US domains (like Minted.com, weddingpaperdivas.com).
I only realized YESTERDAY, that there have been 375 linking domains to my site. I actually thought my SERP dropped b/c of the Panda Algo (which probably still is part of the reason). I had thin/duplicate content, and a cloaking page setup. I've cleaned up those 2 high priority issues.
I've read other postings about Penguin Cleanup, some have noticed a change within a week. But my question is b/c I left this lingering for SO LONG, am I put into a bad position?
-
-
Hey Tyson,
- I'm not quite sure I understand what you mean by 301 redirect. My website is just referenced as a standard direct ahref link on these domains to my site.
In your first point you mentioned you have domain 301ing to your website, if they have spammy link profiles then I would disavow them.
- thanks for the screenshot, where did you compile that profile from?
I compiled that from https://ahrefs.com/
A) The following 3 are "random" directory listings
http://www.skoobe.biz/index.php?go=results&search=enjoy
http://www.incrawler.com/dir/Shopping/Gifts/Weddings/
http://www.directorydump.com/shopping/weddings/These are low quality Directories and you probably will get asked to pay a price for removal. I would just disavow.
B) The following 6 are just page stat type random links.
http://com.stephita.qirina.com/
http://webstarratings.com/ratings-reviews/stephita.com
http://www.kikstat.com/www/stephita.com
https://www.aihitdata.com/company/00872FAC/STEPHITA/overview
http://dig.do/stephita.com
http://www.statsfuse.com/stephita.com.htmlI don't really bother with these or worry about them
C) The following 3 are domains in foreign countries (i.e. Iran, Korea, etc..) I've translated them, but visually it's mumbo jumbo, not an article, just random talk....
http://onvar.ir/term/Invi
http://www.acegreen.co.kr/community/design_news_v.asp?seq=738&intNowPage=1&searchGubun=&searchString=
http://fungood.ir/more/75931_21/Wedding-Invitations-Toronto-Invites-with.htmlDisavow the domains
D) Last set are links that show up in my SEARCH CONSOLE information, but the referring links are "dead", 404 etc... Should I still disavow these broken links?
** I would disavow them if they are spammy just to be on the safe side**
I don't have any "manual actions" in my search console, but from your experience, would you say I might have been hit with a Penguin Penalty? If so, the fact that I've not taken any action until now, does that hurt me even more? (i.e. these could have been up for years, I wouldn't know... :()
Well Penguin rolls out as an update every year or two? We never know as Google rolls it out when they feel like it. If you were hit by Penguin then you would of been hit by it in late 2014.
-
Hi Todd,
Thanks for your input!
-
I'm not quite sure I understand what you mean by 301 redirect. My website is just referenced as a standard direct ahref link on these domains to my site.
-
Makes sense! I figure Google could see more
-
thanks for the screenshot, where did you compile that profile from?
So here is a sample of some of the referring links that are linking to my site: Can you advise me if you would "disavow" these types of links
A) The following 3 are "random" directory listings
http://www.skoobe.biz/index.php?go=results&search=enjoy
http://www.incrawler.com/dir/Shopping/Gifts/Weddings/
http://www.directorydump.com/shopping/weddings/B) The following 6 are just page stat type random links.
http://com.stephita.qirina.com/
http://webstarratings.com/ratings-reviews/stephita.com
http://www.kikstat.com/www/stephita.com
https://www.aihitdata.com/company/00872FAC/STEPHITA/overview
http://dig.do/stephita.com
http://www.statsfuse.com/stephita.com.htmlC) The following 3 are domains in foreign countries (i.e. Iran, Korea, etc..) I've translated them, but visually it's mumbo jumbo, not an article, just random talk....
http://onvar.ir/term/Invi
http://www.acegreen.co.kr/community/design_news_v.asp?seq=738&intNowPage=1&searchGubun=&searchString=
http://fungood.ir/more/75931_21/Wedding-Invitations-Toronto-Invites-with.htmlD) Last set are links that show up in my SEARCH CONSOLE information, but the referring links are "dead", 404 etc... Should I still disavow these broken links?
I appreciate any help on this matter! I don't have any "manual actions" in my search console, but from your experience, would you say I might have been hit with a Penguin Penalty? If so, the fact that I've not taken any action until now, does that hurt me even more? (i.e. these could have been up for years, I wouldn't know... :()
-
-
- If you have a domain 301 redirected to the site with a large spammy link profile then I would disavow the domain.
2. Google console will show more links as they have the largest Index, it's google. Moz Open site explorer just does not crawl as many sites and does not have all the data. Ahrefs crawls and shows the most Data if you want to analyse the links further.
3. If it is just one domain that 301 redirects to your site with lots of spammy links then just Disavow it. Utilize the disavow file
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I remove all vendor links (link farm concerns)?
I have a web site that has been around for a long time. The industry we serve includes many, many small vendors and - back in the day - we decided to allow those vendors to submit their details, including a link to their own web site, for inclusion on our pages. These vendor listings were presented in location (state) pages as well as more granular pages within our industry (we called them "topics). I don't think it's important any more but 100% of the vendors listed were submitted by the vendors themselves, rather than us "hunting down" links for inclusion or automating this in any way. Some of the vendors (I'd guess maybe 10-15%) link back to us but many of these sites are mom-and-pop sites and would have extremely low authority. Today the list of vendors is in the thousands (US only). But the database is old and not maintained in any meaningful way. We have many broken links and I believe, rightly or wrongly, we are considered a link farm by the search engines. The pages on which these vendors are listed use dynamic URLs of the form: \vendors<state>-<topic>. The combination of states and topics means we have hundreds of these pages and they thus form a significant percentage of our pages. And they are garbage 🙂 So, not good.</topic></state> We understand that this model is broken. Our plan is to simply remove these pages (with the list of vendors) from our site. That's a simple fix but I want to be sure we're not doing anything wring here, from an SEO perspective. Is this as simple as that - just removing these page? How much effort should I put into redirecting (301) these removed URLs? For example, I could spend effort making sure that \vendors\California- <topic>(and for all states) goes to a general "topic" page (which still has relevance, but won't have any vendors listed)</topic> I know there is no distinct answer to this, but what expectation should I have about the impact of removing these pages? Would the removal of a large percentage of garbage pages (leaving much better content) be expected to be a major factor in SEO? Anyway, before I go down this path I thought I'd check here in case I miss something. Thoughts?
Intermediate & Advanced SEO | | MarkWill0 -
Will multiple domains from the same company rank for the same keyword search?
I'm trying to convince people that we need good marketing reasons for starting multiple domains, as it will be more difficult to rank multiple sites. Does anyone know if Google actively discourages multiple domains from the same company appearing in the search results for the same keyword? We are creating a separate content website which is related to an existing company website. Would you agree that is best to have these sites on one domain with the content site on a sub-domain perhaps? I'm worried about duplication of effort and cross-keyword targeting in particular. These sites would not have duplicate content.
Intermediate & Advanced SEO | | RG_SEO0 -
Penguin 2.1\. Bad links removed - do I need to wait for next Penguin upgrade to see recovery?
Hi - I have read conflicting advice about this issue - after taking action and removing bad links following a Penguin 2.1 hit, will the site need to wait for the next Penguin upgrade before the link clean-up has any effect? Or will the cleaning of the links be acknowledged and "rewarded" with a ranking improvement before that (assuming all bad links were cleared out)?
Intermediate & Advanced SEO | | StevieD0 -
Penguin Apply To Internal Linking?
Is Penguin focused primarily on backlinks or does it also assess internal linking/anchor text? We've lost about 3,000 visitors a month since the rolling updates were implemented. I'm always careful not to over-react to algo updates but enough time has passed that I think the dust has settled. I try to stay white in all I do but I think if I've over-done anything its the internal linking related products/categories with exact match. My backlink profile also has an over-abundance of affiliate links but that's kind of out of my hands isn't it?
Intermediate & Advanced SEO | | AWCthreads0 -
Sub domain on root domain
Hello,
Intermediate & Advanced SEO | | dror999
I have a question that I can't find a good answer on.
I have a site, actually a "portal"/ "directory" for service providers.
Now, for start, we opened every service provider own page on our site, but now we get a lot of applications from those providers that thy want sites from their own.
We want to make every service provider his own site, but on sub domain url. ( they don’t mind… its ok for them)
So, my site is www.exaple.com
There site will be: provider.exaple.com
Now I have two questions:
1. can it harm my site in SEO?
2. if one from those sub domain , punished by google because is owner do "black hat seo" , how it will affect the rood domin? It can make the root domain to get punished?
Thanks!!0 -
Guest Blogging Results - domain authority vs. page authority -
I have been guest blogging on various websites. The web pages that display my guest posts have zero page authority but a great domain authority of 60/100 or more. 2 links I developed are noted below as examples: http://www.evancarmichael.com/Small-Business-Consulting/8101/How-To-Hire-Extraordinary-Salespeople.html https://www.biggerpockets.com/blogs/4195/blog_posts/28772-4-essentials-for-rental-property-owners Am I wasting my time with this?
Intermediate & Advanced SEO | | superZj
Thank you!0 -
Move webshop domain to the brand domain?
Hello, A client of mine has a brand with a website for over 10 years now.
Intermediate & Advanced SEO | | Seeders
About 4 years ago the have opened a webshop on an other domain (like www.brandnamewebshop.com). At this moment the brand domain has a seomoz authority of 45.
The webshop domain authority is 25. The question:
Would it not be better to transfer the webshop to the brand domain because of the domain authority? If so, how can this be done the best way? With a 301?
I also think: what a loss of energy of building the authority on the other domain.
Is it an idea to use both domains for a webshop and rewrite the content? Or is there an other way to still make use of the built up domain authority? Would it really help the other domain when I make a 301 redirect (and make use of the pointing links to the webshop domain?). I hope somebody have some experience with this...
Looking forward to the possibilities! Gerjan0 -
How to detect a bad neighborhood links?
I have the feeling that I am suffering from negative seo, so there is a way to get a list of links that should remove in the google disavow links tool ?
Intermediate & Advanced SEO | | Valarlf0
