Bad Domain Links - Penguin? - Moz vs. Search Console Stats?
-
I've been trying to figure out why my site www.stephita.com has lost it's google ranking the past few years. I had originally thought it was due to the Panda updates, but now I'm concerned it might be because of the Penguin update. Hard for me to pinpoint, as I haven't been actively looking at my traffic stats the past years.
-
So here's what I just noticed. On my Google Search Console - Links to your Site, I discovered there are 301 domains, where over 75% seem to be spammy. I didn't actively create those links.
-
I'm using the MOZ - Open site Explorer tool to audit my site, and I noticed there is a smaller set of LINKING DOMAINS, at about 70 right now. Is there a reason, why MOZ wouldn't necessarily find all 300 domains?
-
What's the BEST way to clean this up??? I saw there's a DISAVOW option in the Google Search Console, but it states it's not the best way, as I should be contacting the webmasters of all the domains, which is I assume impossible to get a real person on the other end to REMOVE these link references.
HELP!
What should I do? -
-
It's better to 301 a page if it's relevant than it is to let it go 404. However if it's a service or product you no longer carry, might be better off keeping it a 404 so the page goes away unless you think you'll be using that url again.
404s won't hurt you unless you have a huge number of them, but if you have a page you no longer want and wish for it to be forgotten about, you can let it 404 and will eventually go bye bye.
-
I don't know how that .js subpage is looping in the address bar. I'm just using the foundation.zurb.com code to develop my site as mobile friendly.
I do have an idea why the articles page might be spinning out of control. It's a dynamic page that pulls my articles content, and it's dependent on the URL, so technically I could have an infinite number of pages spawning. I will correct the script to realize this error, and have it redirect to a base page.
As for the other previous cloaking pages, I'm assuming if I do a 301 redirect to the homepage, that would be better than to DELETE the page entirely which would return a 404.
-
I ran your site url through Screaming Frog it found more pages than I expected at 30% and was still counting so I took a quick glance at the urls it was returning and found hundreds of urls containing the same words like " vendor " , " foundation " , " js " over and over and over again. Then just did a search for " .js subpage looping ".
About moz saying what your page count was, this is what threw me off, since after I saw the ungodly high amount of urls I did a site:yourdomain.com search in google to see how many pages were being indexed and was returned a normal amount somewhere around 1,240 pages.
The cloaking / anything blackhat is a big deal to get off as soon as possible, even the left over script could be seen as an issue. But in this case it's also causing extra pages to be seen so removing that script will help clear up a great bit of google's confusion, since if screaming frog was having issues so was google then, and could even be why you dropped in ranking.
Screaming frog saw these pages as various different ones:
All from the article subpage, so I guess that's the home of that script or at least where it starts to make new pages.
-
Thank for helping out with the site audit! I truly appreciate it.
How did you detect the loop in the link structure?
Regarding the page count, the current count I thought was < 1000 based on MOZ Site Crawl tool.
** Prior to January of this year (2016), I had a cloaking page that was running that basically generated pages with a different city reference. So my site came up for searches like "Wedding invitations Banff, W I Timbuktuu, W I any city". Obviously this was a black-hat tactic, which I figured out a few months ago is a NO-NO with the PANDA update. So I unreferenced the link from my link structure, and I also requested that the directory structure be removed from the Google Webmaster Tools. I just realized I didn't take that script offline, or have it re-direct - i.e. www.stephita.com/wedding/type_anything_here is still live. But there is no LINK from my site that would go there anymore. ** I'm thinking I should probably do a 301 redirect to my homepage instead now.
So I'm curious as to how Screaming Frog is seeing these pages, if it is indeed these pages. www.stephita.com/wedding/____________________
Thanks!
-
***Seems something is going on with your internal link structure, many many many weird urls with duplicate subpages:
Seems it's a .js subpage loop going on, http://wordpress.stackexchange.com/questions/93844/child-pages-loop
So that handles the huge page total issue, get this handled and maybe peek into other pages and see how they are worded and structured, make sure every page has it's h1 tag but only one h1, h2 - h5 for anything else if it needs it. For a 10 position move, I wouldn't imagine there is a huge number of issues going on. It's really a matter of finding them.
End Update
Curious as I went to run screaming frog on your site and its saying it's only 30% done crawling your site at 8,549 pages, then is showing me 33,359 pages and counting to go!
Do you really have this many pages?
If so, do you need them all?
If you have 30k + pages, odds are you might have some bad pages from all that. Having this amount of pages makes it very hard to do good SEO unless you've been doing good SEO since day one.
I'd say it's not a penalty since you only went from page 1 to page 2, that happens even on good days, often is a sign of competitors one uping you somehow.
If you can, I'd suggest pruning a good portion of these pages unless you have to have them, like they are product pages or such.
At the time of writing this, it's still totaling your total pages it's up to 53,932 pages now and still counting, screaming frog is at 29% to go from 31%.
-
Oh.. I was hoping it was a 'penalty' b/c at least there is hope that fixing those issues would make Google smile at my site again
I've basically started this 'clean-up' process the past 3 months, where I've used the MOZ tools to identify "high/medium priorities".I've put in the effort to remove all "duplicate pages" which were marked as HIGH, and I've adjusted the META TITLES which were marked as MEDIUM. According to MOZ crawl report, I'm down to a few issues vs. 900 or so when I first ran the report a few months ago.
As for the Penguin fixes, I only discovered this 3 days ago, and submitted my first DISAVOW list yesterday which contained 900+ urls and about 210 domains.
** If the 10 placement drop wasn't a penalty, I can only hope this 'clean-up' attempt can only help? But the big question is timeframe...
Is it possible Google would have only assessed my site a MINOR penalty? Or is it either nothing or SEVERE? -
As for the article about penguin being part of the core, I have read a few articles back in January of this year about that but nothing instead it was an unnamed update and then the adwords update. So far no one has 100% declared that penguin was a part of the core update yet.
And from what you've mention a 10 placement drop doesn't seem like a algo penalty, more like some tactics that were working on your site stopped, or other sites have updated their content and appear more relevant, causing your site to drop to page 2.
Let me give an example as to what penguin penalty could be like, I have a keyword for a client that would not rank passed page 4, and has grown to stay on page 6 and 7. After a recent site redesign where I made SEO top priority when I launched the site, about 2 weeks or so after we saw that keyword rise, from page 7 up to page 3. Stayed on page 3 for a day or two, then dropped back to page 6 then to page 7, stopping right back where it was. This most likely is a penguin penalty, especially after seeing a not so awesome backlink profile with excessive anchor text for that keyword to directories galore as well as comment spam.
I think if you were hit by an algo penalty that you'd be suffering ranking issues on a much more severe scale. I don't think your site sucks, it's just that you always have to keep your site up to date, gone are the days of posting content and walking away. You need to creating new content, promote it correctly, improve it, constantly check backlinks, check competitors, stay on top of current trends of your industry. And changes you make today, won't really show results in SERPs as fast as you'd like. Sometimes it can take time.
-
You are soooo right about the scouring the internet till I'm cross-eyed!
hahaahI read this article: http://www.brafton.com/news/seo-1/penguin-to-join-googles-core-algorithm-what-you-need-to-know/ That 'expert' says Penguin has already been put into the core algorithm. ** I can't seem to find anything that "Google" has said, do you know?
Here's a quick rundown, my site www.stephita.com, used to rank on near the top of page 1 for google.ca on search term "wedding invitations", some time ago (I haven't been keeping track, but I believe maybe 1-2 years ago probably) my site has dropped to page middle of page 2. So I'm thinking my site got dropped about 10 spaces. If I was hit with any penalties b/c of Panda & Penguin, does a 10 place drop seem reasonable, or would it have been more? A few facts:
-
I never got a manual action
-
I believe Panda penalty b/c I had some 'black-hat seo' pages, like a cloaking page, and also some relatively THIN content.
-
I believe Penguin b/c I just did a backlink audit, and felt I had about 200 spammy domains linking to me which amounted to about 900 urls.
So with those 3 points, does 10 place drop seem reasonable, or does my site just suck
hahaha -
-
Panda is part of the google core algorithm now, penguin is not yet, but is expected soon ( sometime between now and the end of this year soon )
So since penguin isn't yet part of the core algorithm, the key is to submit your disavow list as soon as possible, since it can take anywhere from a month to three months for google to nofollow those links you sent. They start nofollowing the urls once you send it in, but for reasons I have yet to gather aren't as speedy with this process as it is when you use the google fetch feature.
My understanding of how site' recover now from penalties are two fold.
If it's a manual penalty ( such as they get a message from google in their webmaster console ) you fix what they mention and submit a reconsideration request.
If it's a algorithmic penalty, you scour the internet and read as much as you can until you get a little crosseyed then you read some more, fix the issues that relate to your penalty ( or what you suspect is your penalty ) and then you wait, you tackle other aspects of your site's SEO and wait until that specific algorithm is rerun / updated and run, then it's not even 100% that your site will recover right away, might take a bit of time after that.
So once you do recover, stress to your client, your boss, or yourself that this cannot ever happen again, and to never ever ever neglect your site's SEO again or this issue will only be 100 times worse.
Hope this helps!
-
Thanks for the input!
What's your understanding of how sites recover from penalties now? I've seen a posting about how Panda and Penguin are now 'baked in' the search algorithm. Where as in the past, sites would have to wait till Google rolled out a Panda/Penguin update for sites to show any 'recovery'. So, if it is 'baked in', if I submitted my 'disavow' list a few hours ago, should I hopefully see immediate results now?
Thanks!
-
If those links aren't doing anything else but redirecting, yes, get rid of those, they aren't helping you and could hurt you if they haven't already.
If they were all from the same domain, i'd disavow that domain as well, but that doesn't seem to be the case. As it was mentioned above by another poster.
You shouldn't need to disavow eventwire.com unless there is a backlink from them to your site, and it's not helping anyone. But remember not to get carried away when using the disavow tool, just remember this nifty saying **" when in doubt, leave it out. when it stinks, disavow the links " .
** -
No one at google reads these, it's all done automatically now, the comments are for your usage when you look at the disavow list at a later date.
Yes, when you submit your next disavow file, say you disavowed 10 the 1st time and wanted to disavow 10 more, so your 2nd disavow file will have 20 urls in it, 10 from the last time and 10 from this time.
It's best course to download the text file from before and add the urls you want to remove this time to that txt file, add a comment about that time to help you remember or inform others.
Here is an article from 2014 from moz.com that helps explain this further - https://moz.com/blog/guide-to-googles-disavow-tool
Remember you don't have to comment these, but at the very least, comment each time you use the disavow feature at least, this will make things easier on you.
Just be careful, removing even spammy urls could alter your domain authority or possibly worse. always be 100% sure if you want the urls selected to be ignored, this process can take as little as a month and as much as 3 months with some people taking a year to have a url disavowed. And if you make a mistake, that's about how long it can take to correct it.
-
-
When submitting a disavow list, should I put a COMMENT on each one? Does someone at Google read it?
-
If I send 10 at a time, when I send the second batch, should I re-include the first batch with it? So basically just upload the same file over and over with new additions.
-
-
Just a " heads Up " the Original Poster was saying he had " 301 domains linking to his site, not that he has a domain 301ing to his site "
I've just come from this area for a recent client, Luckily for me it was only 700 urls total.
**I got all the links linking back to my site ( backlinks ) from: **
- Google Webmaster Tools ( Search console )
- Moz.com
- Ahrefs.com
- SEMrush
Added them all to a spreadsheet, downloaded an addon in google sheets called "remove duplicates" to remove same urls, once that was done, I put as much information for these urls as I could find, both what was already listed from those sites I got the backlinks from. Then I also went and visited every single one of them, listed if they were still working or 404, what the anchor text was, what the domain authority and page authority was and anything else to help, like was it a foreign site or a directory listing or comment spam.
I had over 75% of my backlink profile was nofollow or was a directory / comment spam.
So far I have disavowed 153 urls, I have more but want to do chunks at a time so if something really horrid and unforeseen happens I can reavow ( yes thats a real thing ) the urls a section at a time a wait.
I recommend doing this once to create a master backlink list, and just keep things dated and correct so you only have to add to it each month and keep up to date on your backlink profile!
Since backlinks are on par with content in terms of ranking there is no reason to not be so involved with backlinks.
-
Hi Todd,
Thanks again.
-
In your opinion, I should disavow those low quality directory listings? Would Google/Bing consider those as SPAM?
-
I thought those "stat" type referring links look more spammy then directory listings. Would these have "hurt" my ranking at all?
-
I've looked at my Google Analytic profile, and have noticed a steep decline the past 2 years. The fact that some of these links might have been there for a while, does cleaning it up 'now' greatly improve my standing?
So the drop in my SERP is based on users doing a keyword search for the term "wedding invitations", on google.ca. I started this company about 14 years ago, and pretty sure my domain has been on page 1 for the first 10-12 years. Especially in my local market in Toronto, we were top 3 results. I've only noticed earlier this year, that my site has dropped to page 2! (it probably dropped to page 2 much earlier, which probably coincides with my traffic drop, so let's say it has been on page 2 for the past 2 years). There are now 4 other local based companies that sit ahead of my site, along with some large US domains (like Minted.com, weddingpaperdivas.com).
I only realized YESTERDAY, that there have been 375 linking domains to my site. I actually thought my SERP dropped b/c of the Panda Algo (which probably still is part of the reason). I had thin/duplicate content, and a cloaking page setup. I've cleaned up those 2 high priority issues.
I've read other postings about Penguin Cleanup, some have noticed a change within a week. But my question is b/c I left this lingering for SO LONG, am I put into a bad position?
-
-
Hey Tyson,
- I'm not quite sure I understand what you mean by 301 redirect. My website is just referenced as a standard direct ahref link on these domains to my site.
In your first point you mentioned you have domain 301ing to your website, if they have spammy link profiles then I would disavow them.
- thanks for the screenshot, where did you compile that profile from?
I compiled that from https://ahrefs.com/
A) The following 3 are "random" directory listings
http://www.skoobe.biz/index.php?go=results&search=enjoy
http://www.incrawler.com/dir/Shopping/Gifts/Weddings/
http://www.directorydump.com/shopping/weddings/These are low quality Directories and you probably will get asked to pay a price for removal. I would just disavow.
B) The following 6 are just page stat type random links.
http://com.stephita.qirina.com/
http://webstarratings.com/ratings-reviews/stephita.com
http://www.kikstat.com/www/stephita.com
https://www.aihitdata.com/company/00872FAC/STEPHITA/overview
http://dig.do/stephita.com
http://www.statsfuse.com/stephita.com.htmlI don't really bother with these or worry about them
C) The following 3 are domains in foreign countries (i.e. Iran, Korea, etc..) I've translated them, but visually it's mumbo jumbo, not an article, just random talk....
http://onvar.ir/term/Invi
http://www.acegreen.co.kr/community/design_news_v.asp?seq=738&intNowPage=1&searchGubun=&searchString=
http://fungood.ir/more/75931_21/Wedding-Invitations-Toronto-Invites-with.htmlDisavow the domains
D) Last set are links that show up in my SEARCH CONSOLE information, but the referring links are "dead", 404 etc... Should I still disavow these broken links?
** I would disavow them if they are spammy just to be on the safe side**
I don't have any "manual actions" in my search console, but from your experience, would you say I might have been hit with a Penguin Penalty? If so, the fact that I've not taken any action until now, does that hurt me even more? (i.e. these could have been up for years, I wouldn't know... :()
Well Penguin rolls out as an update every year or two? We never know as Google rolls it out when they feel like it. If you were hit by Penguin then you would of been hit by it in late 2014.
-
Hi Todd,
Thanks for your input!
-
I'm not quite sure I understand what you mean by 301 redirect. My website is just referenced as a standard direct ahref link on these domains to my site.
-
Makes sense! I figure Google could see more
-
thanks for the screenshot, where did you compile that profile from?
So here is a sample of some of the referring links that are linking to my site: Can you advise me if you would "disavow" these types of links
A) The following 3 are "random" directory listings
http://www.skoobe.biz/index.php?go=results&search=enjoy
http://www.incrawler.com/dir/Shopping/Gifts/Weddings/
http://www.directorydump.com/shopping/weddings/B) The following 6 are just page stat type random links.
http://com.stephita.qirina.com/
http://webstarratings.com/ratings-reviews/stephita.com
http://www.kikstat.com/www/stephita.com
https://www.aihitdata.com/company/00872FAC/STEPHITA/overview
http://dig.do/stephita.com
http://www.statsfuse.com/stephita.com.htmlC) The following 3 are domains in foreign countries (i.e. Iran, Korea, etc..) I've translated them, but visually it's mumbo jumbo, not an article, just random talk....
http://onvar.ir/term/Invi
http://www.acegreen.co.kr/community/design_news_v.asp?seq=738&intNowPage=1&searchGubun=&searchString=
http://fungood.ir/more/75931_21/Wedding-Invitations-Toronto-Invites-with.htmlD) Last set are links that show up in my SEARCH CONSOLE information, but the referring links are "dead", 404 etc... Should I still disavow these broken links?
I appreciate any help on this matter! I don't have any "manual actions" in my search console, but from your experience, would you say I might have been hit with a Penguin Penalty? If so, the fact that I've not taken any action until now, does that hurt me even more? (i.e. these could have been up for years, I wouldn't know... :()
-
-
- If you have a domain 301 redirected to the site with a large spammy link profile then I would disavow the domain.
2. Google console will show more links as they have the largest Index, it's google. Moz Open site explorer just does not crawl as many sites and does not have all the data. Ahrefs crawls and shows the most Data if you want to analyse the links further.
3. If it is just one domain that 301 redirects to your site with lots of spammy links then just Disavow it. Utilize the disavow file
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
More bad links
Hi, After a recent disastrous dalliance with a rogue SEO company I disavowed quite a few domains (links he had gained) which I was receiving a penalty of about 23 places. I cleaned up the site and added meta descriptions where missing, and deleted duplicate titles and pages. This gained me another 5 places. In the meantime I have been getting a few links from wedding blogs, adobe forums and other relevant sites so was expecting an upward momentum. Since the high point of bottom of page 1 I have slowly slid back down to near the bottom of page two for my main keywords. Just checked my webmaster tools latest links and another 4 domains have appeared (gained by the dodgy SEO) : domain:erwinskee.blog.co.uk domain:grencholerz.blog.co.uk domain:valeriiees.blog.co.uk domain:gb.bizin.eu They all look bad so I am going to disavow. I expect to find an improvement when I disavow these new domains. As I have said, have started using the open site explorer tool to check my competitors backlinks and getting some low level links(I'm a wedding photographer) like forum comments and blog comments and good directories. I know there is much more than this to SEO and plan on raising my game as time progresses. I have also gained more links from the domains I disavowed on the 8th January mostly from www.friendfeed.com. will webmaster tools ignore any new links from previously disavowed domains? Like I have said I know there are better ways to get links, but are these links (forum comments, blog comments and respectable directories) one way of raising my rankings? To be honest that is all my competitors have got other than some of the top boys might have a photograph or two on another site with a link. No-one has a decent article or review anywhere (which is my next stage of getting links). Thanks! David.
Intermediate & Advanced SEO | | WallerD0 -
Some site's links look different on google search. For example Games.com › Flash games › Decoration games How can we do our url's like this?
For example Games.com › Flash games › Decoration games How can we do our url's like this?
Intermediate & Advanced SEO | | lutfigunduz0 -
Sub domain vs. sub folder
I know this has probably been asked many times and answered too, but things change a lot, so I would like to know with current search engine algos and co. The scenario is as follows: Building an ecommerce site and also want to incorporate a Q&A section, for support and FAQ's and such. should we go ahead and sub domain this like: community.test.com or rater go with test.com/community. I would really like to know why, why not and maybe some real life examples. Thank you all
Intermediate & Advanced SEO | | s-s0 -
Linking and non-linking root domains
Hi, Is there any affect on SEO based on the ratio of linking root domains to non-linking root domains and if so what is the affect? Thanks
Intermediate & Advanced SEO | | halloranc0 -
Redirect ruined domain to new domain without passing link juice
A new client has a domain which has been hammered by bad links, updates etc and it's basically on its arse because of previous SEO guys. They have various domains for their business (brand.com, brand.co.uk) and want to use a fresh domain and take it from there. Their current domain is brand.com (the ruined one). They're not bothered about the rankings for brand.com but they want to redirect brand.com to brand.co.uk so that previous clients can find them easily. Would a 302 redirect work for this? I don't want to set up a 301 redirect as I don't want any of the crappy links pointing across. Thanks!
Intermediate & Advanced SEO | | jasonwdexter0 -
Image Links Vs. Text Links, Questions About PR & Anchor Text Value
I am searching for testing results to find out the value of text links versus image links with alt text. Do any of you have testing results that can answer or discuss these questions? If 2 separate pages on the same domain were to have the same Page Authority, same amount of internal and external links and virtually carry the same strength and the location of the image or text link is in the same spot on both pages, in the middle of the body within paragraphs. Would an image link with alt text pass the same amount of Page Authority and PR as a text link? Would an image link with alt text pass the same amount of textual value as a text link? For example, if the alt text on the image on one page said "nike shoes" and the text link on the other page said "nike shoes" would both pass the same value to drive up the rankings of the page for "nike shoes"? Would a link wrapped around an image and text phrase be better than creating 2 links, one around the image and one around the text pointing to the same page? The following questions have to do with when you have an image and text link on a page right next to each other, like when you link a compelling graphic image to a category page and then list a text link underneath it to pass text link value to the linked-to page. If the image link displays before the text link pointing to a page, would first link priority use the alt text and not even apply the anchor text phrase to the linked page? Would it be best to link the image and text phrase together pointing to the product page to decrease the link count on the page, thus allowing for more page rank and page authority to pass to other pages that are being linked to on the page? And would this also pass anchor text value to the link-to page since the link would include an image and text? I know that the questions sound a bit repetitive, so please let me know if you need any further clarification. I'd like to solve these to further look into ways to improve some user experience aspects while optimizing the link strength on each page at the same time. Thanks!
Intermediate & Advanced SEO | | abernhardt
Andrew0 -
Exact match domain or root domain for speedy SEO?
I am doing SEO for a website that has constantly rotating and only temporarily pertinent subjects on it. Let's say these information and subject cycles go for about 6 months. Assuming this would it be more effective to optimize exact match domains for each 6 month cycle or make a main domain with a few of the keywords and just target a page for each roaming subject? Advantage of the subject is I get domain authority to feed off of, advantage of the exact match is, of course exact match domains are a powerful tool to rank highly and it is only a medium competitive market, usually about 40 domain and page authority. What do you guys think? Do you have any techniques to dominate temporary and rotating markets?
Intermediate & Advanced SEO | | MarloSchneider0 -
Robots.txt: Link Juice vs. Crawl Budget vs. Content 'Depth'
I run a quality vertical search engine. About 6 months ago we had a problem with our sitemaps, which resulted in most of our pages getting tossed out of Google's index. As part of the response, we put a bunch of robots.txt restrictions in place in our search results to prevent Google from crawling through pagination links and other parameter based variants of our results (sort order, etc). The idea was to 'preserve crawl budget' in order to speed the rate at which Google could get our millions of pages back in the index by focusing attention/resources on the right pages. The pages are back in the index now (and have been for a while), and the restrictions have stayed in place since that time. But, in doing a little SEOMoz reading this morning, I came to wonder whether that approach may now be harming us... http://www.seomoz.org/blog/restricting-robot-access-for-improved-seo
Intermediate & Advanced SEO | | kurus
http://www.seomoz.org/blog/serious-robotstxt-misuse-high-impact-solutions Specifically, I'm concerned that a) we're blocking the flow of link juice and that b) by preventing Google from crawling the full depth of our search results (i.e. pages >1), we may be making our site wrongfully look 'thin'. With respect to b), we've been hit by Panda and have been implementing plenty of changes to improve engagement, eliminate inadvertently low quality pages, etc, but we have yet to find 'the fix'... Thoughts? Kurus0