Broken Links from Open Site Explorer
-
I am trying to find broken internal links within my site. I found a page that was non-existent but had a bunch of internal links pointing to that page, so I ran an Open Site Explorer report for that URL, but it's limited to 25 URLs.
Is there a way to get a report of all of my internal pages that link to this invalid URL? I tried using the link: search modifier in Google, but that shows no responses.
-
Whew! Big thread.
Sometimes, when you can't find all the broken links to a page, it's easier simply to 301 redirect the page to a destination of your choice. This helps preserve link equity, even for those broken links you can't find on large sites. (and external links, as well)
Not sure if this would help in your situation, but I hope you're getting things sorted out!
-
Jesse,
That's where I started my search, but GWMT wasn't showing this link. I can only presume that because it isn't coming back a 404 (it is showing that "We're Sorry" message instead) that they're considering that message to be content.
Thanks!
-
Lynn, that was a BIG help. I had been running that report, but was restricted to 25 responses. When I saw your suggestion to filter for only internal links, I was able to see all 127.
Big props. Thanks!
-
One more thing to add - GWMT should report all 404 links and their location/referrer.
-
oops! i did not know this. Thanks Irving.
-
Use the word FREE with an asterisk because sreaming frog is now limiting the free version to 500 pages. Xenu is better, even brokenlinkcheck.com lets you spider 3000 pages.
500 pages makes the tool practically worthless for any site of decent size.
-
Indeed if it is not showing a 404, that makes things a bit difficult!
You could try another way, use OSE!
Use the exact page, filter for only internal links, boom 127 pages that link to it. There might be more, but this should get you going!
-
Jesse:
I appreciate your feedback, but am surprised that the ScreamingFrog report found no 404s. SEOmoz found 15 in Roger's last crawl, but those aren't the ones that I'm currently trying to solve.
The problem page is actually showing up as duplicate content, which is kinda screwy. When visiting the page, our normal 404 error doesn't appear (which our developers are still trying to figure out), but instead, an error message appears:
http://www.gallerydirect.com/about-us/media-birchwood
If this were a normal 404 page, we'd probably be able to find the links faster.
-
I got tired of the confusion and went ahead and proved it. Not sure if this is the site you wanted results for, but I used the site linked in your profile (www.gallerydirect.com)
took me about 90 seconds and I had a full list... no 404s though
anyway here's a screenshot to prove it:
http://gyazo.com/67b5763e30722a334f3970643798ca62.png
so what's the problem? want me to crawl the fbi site next?
-
I understand. Thing is, there is a way and the spider doesn't affect anything. Like I said, I have screaming frog installed on my computer and I could run a report for your website right now and you or your IT department would never know it happened.. I just don't understand the part where the software doesn't work for you but to each their own i suppose.
-
Jesse:
That movie was creepy, but John Goodman was awesome in it.
I started this thread because I was frustrated that OSE restricts my results to 25 links, and I simply wanted to find the rest for that particular URL. I was assuming that there was either:
a. A method for getting the rest of the links that Roger found
b. Another way of pulling these reports from someone who already spiders them (since I can't get any using the link:[URL] in Google and Webmaster Tools isn't showing them).
Thanks to all for your suggestions.
-
run the spider based app from outside their "precious network" then. hell, i could run it right now for you from my computer at work if I wanted. Use your laptop or home computer. It's a simple spider you don't have to be within any network to run it. You could run one for CNN.com if you'd like as well...
-
How else do you expect to trace these broken links without using a "spider?"
Obviously it's the solution. And the programs take up all of like 8 megs... so what's the problem/concern?
I second the screaming frog solution. It will tell you exactly what you need to know and has ZERO risk involved (or whatever it is that's hanging you up). The bazooka comparison is ridiculous, because a bazooka destroys your house. Do you really think a spider crawl will affect your website?
Spiders crawl your site and report findings. This happens often whether you download a simple piece of software or not. What do you think OSE is? Or Google?
I guess what we're saying is if you don't like the answer, then so be it. But that's the answer.
PS - OSE uses a spider to crawl your site...
PPS - Do you suffer from arachnophobia? That movie was friggin awesome now I want to watch old Jeff Daniels films.
PPSS - Do you guys remember John Goodman being in that movie? Wow the late 80s early 90s were really somethin' special.
-
John, I certainly see your point, but our IT guys would not take too kindly to me running a spider-based app from inside their precious network, which is why I was looking for a less intrusive solution.
I'm planning on a campaign to revive "flummoxed" to the everyday lexicon next.
-
Hi Darin,
Both these softwares are made for exactly this kind of job and they are not huge system killing programs or anything. Seriously I use one or both almost every day. I suggest downloading them and seeing how you go, I think you will be happy enough with the results.
-
The way I see it, its much like you missing the last flight home, and you have a choice of getting the bus, that means you might take a little longer, or of course you can wait for the next flight ,which happens to be tomorrow evening, the bus will get you home that night.
I get the bus each and every time, I get home, later than expected I grant you, but I get home a lot quicker than waiting for the plane tomorrow.
Bewildered, I didn't realise it had fallen out of the diction, its a common word (I think) in Ireland, oh and I am still young (ish)
-
John:
Bewildered. There's a good word that I'm happy to see someone is keeping it alive for the younger generations.
I'm not ungrateful for your suggestions, but both involve downloading and installing a spider, which seems like overkill, much like using a bazooka to kill a housefly.
-
I am bewildered by this, I have told you one, Lynn has told you another piece of free software that will do this for you.
Anyway, good luck with however you resolve our issues
-
Lynn, part of the problem is definitely template-based, and one of our developers is working on that fix now. However, I also found a number of non-template created links to this page simply due to UBD error (an old Cobol programming term meaning User Brain Dead).
I need to find all of the non-template based, UBD links that may have been created and fix them.
-
Xenu will also do a similar job and doesn't have a limit which I recall the free version of screaming frog has: http://home.snafu.de/tilman/xenulink.html
If you have loads of links to this missing page it sounds like you maybe have a template problem with the links getting inserted on every or lots of pages. In that case if you find the point in the template you will have fixed them all at once (if indeed it is like this).
-
Darin
Its a stand alone piece of software you run, it crawls your website and finds out broken inbound, outbound or internal links, tells you them ,you go and fix them
Enter your URL, be it a page or directory, run it, it will give you all bad links. And it wont limit you to 25.
You don't need to implement anything ... run the software once, use it, and well bin it afterwards if you wish
But by all means, you can do as you suggest with SE ...
Regards
John
-
John,
While I could look at implementing such a spider to run the check sitewide on a regular basis, I am not looking to go that far at the moment. For right now, I'm only looking for all of the pages on my site that link to a single incorrect URL. I would have to think that there's a solution available for such a limited search.
If I have to, I suppose I can fix the 25 that Open Site Explorer displays, wait a few days for the crawler to run again, then run the report again, fix the next 25, then so on and so on, but that's going to spread the fix out potentially over a number of weeks.
-
Free tool, non SEO Moz related
http://www.screamingfrog.co.uk/seo-spider/ , run that, will find all broken links, where they are coming from etc etc
Hope I aint braking any rules posting it
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does Navigation Bar have an effect on the link juice and the number of internal links?
Hi Moz community, I am getting the "Avoid Too Many Internal Links" error from Moz for most of my pages and Google declared the max number as 100 internal links. However, most of my pages can't have internal links less than 100, since it is a commercial website and there are many categories that I have to show to my visitors by using the drop down navigation bar. Without counting the links in the navigation bar, the number of internal links is below 100. I am wondering if the navigation bar links affect the link juice and counted as internal links by Google. The Same question also applies to the links in the footer. Additionally, how about the products? I have hundreds of products in the category pages and even though I use pagination I still have many links in the category pages (probably more than 100 without even counting the navigation bar links). Does Google count the product links as internal links and how about the effect on the link juice? Here is the website if you want to take a look: http://www.goldstore.com.tr Thank you for your answers.
Intermediate & Advanced SEO | | onurcan-ikiz0 -
Our parent company has included their sitemap links in our robots.txt file - will that have an impact on the way our site is crawled?
Our parent company has included their sitemap links in our robots.txt file. All of their sitemap links are on a different domain and I'm wondering if this will have any impact on our searchability or potential rankings.
Intermediate & Advanced SEO | | tsmith1310 -
How to proceed? Older ecommerce site, unnatural link warning 2013, disavow, now what?
Hello all, I have a small, older ecommerce site. It has been around since 2002. It ranked very well until a few years ago. It currently does rank for some terms, but not many. (I am trying to say that it is not completely off the map.) Our domain authority is 36. Our Spam Score in Open Site Explorer is a 2/10. We received a notice in GWT in May 2013 re: unnatural links. That notice has since cleared from our account. I assume that it has expired. We were working with an SEO consultant when we received the notice from Google in 2013. He started working on cleaning up our link profile at that point. He submitted a disavowal file to Google with all of the domains that he was not able to get cleaned up manually. He kept working and updated the file again in June 2014. He told me that we did not have to file a reconsideration request. He did find that an SEO company that I hired in the past had gotten me a lot of spammy links. We got these taken down. There are still some spammy links that seem to keep cropping up. I have started going through Open Site Explorer to again contact some of these spammy sites to ask them to take our links down. Of course, the emails immediately bounce back to me. I am documenting everything. I feel like I am in a hole and can't dig out. What am a doing wrong? Should I disavow again? Should we have filed a reconsideration request a year or two ago? At this point, is it too late to do so as the penalty no longer shows up in my GWT account? How should I proceed? I prefer not to post my URL, but I would be happy to PM it to anyone who can offer advice. Thanks in advance. Melissa
Intermediate & Advanced SEO | | pajamalady0 -
Site Wide Footer Links Exception, Any Advice ?
I was reading the following Q&A on site wide footer links, http://moz.com/community/q/site-wide-links-from-another-domain-could-these-cause-a-problem I feel my situation is slightly different however,we have lots of international sites linking to each other through these links like our sites for different counties and languages so our German, French and Spanish sites, http://www.cirrusresearch.co.uk/ Our main UK site has always ranked very well and has never really had a problem despite always having had these followed sitewide footer links, Because of this we regularly get high amount of visitors performing English language searches from different counties and i don't think it is a bad thing having more country/language specific sites of ours available in the footer for visitors that may prefer a more localized site, Our main website has to be at least 10+ years old at least, has a lot of strong links compared to our competitors, but the smaller German and Spanish sites are relatively smaller in size and most only 1-2 years old, my big fear is that these smaller sites would not be able to stand on there own without these footer links from our main site, After reading the community question caused me to question this ?, should i take a leap of faith and no-follow all of these site wide footer links connecting all of our sites ? we never really had a problem ranking so i don't really see the need but would this be the best thing to do ? Thank you, James
Intermediate & Advanced SEO | | Antony_Towle0 -
Is this site legit?
http://www.gglpls.com/ is this site legit? Submit website to google + directory?
Intermediate & Advanced SEO | | SEODinosaur0 -
Link Building Ideas for a health site
Hi, I am trying to rank a health related website. This is the url: www.ridpiles.com Domain age is 1 year 6 months. Done Directory submissions Blog Comments + Forum posts Done Social Bookmarks Article submissions (Not much) I have done competitor analysis. All of my competitors are just had links from directories and some link exchanges. They got links from quality sites like Yahoo dir. I know my site is far better than my competitors and has 100% unique content. I have submitted to yahoo directory inclusion, but still no luck i hadn't accepted into it. I am planning to go for a sponsered review but dont know, weather the link will be valuable for that much of money. I was left with Guest Blogging. I see this is the only option for me to build links. But i have a very tough competiton, i must compete with most reputed sites like webmd.com etc, i need to get more good links. But i cant get what other ways to get authoritative links. If Guest blogging is the only option for me, how many posts do i need to do daily? And can someone suggest me good Guest blogging sites? Anyhelp would be appreciated.
Intermediate & Advanced SEO | | Indexxess0 -
Big Site Wide Link
Hi Guys, I've noticed that Google is starting to de-value site-wide links... Our previous SEO agency sourced us a site wide link on a big website and at the moment within Google Webmaster Tools its showing 749,726 links from this 1 source. Do you think this is too many? Could this be being flagged by Google? Here is the site: http://tinyurl.com/7bttw3b Cheers, Scott
Intermediate & Advanced SEO | | ScottBaxterWW0 -
Has anyone found a way to get site links in the SERPs?
I am wanting to get some site links in the serps to increase the size of my "space", has anyone found a way of getting them? I know google says that its automatic and only generated if they feel it would benifit browsers but there must be a rule of thumb to follow. I was thinking down the line of a tight catagorical system that is implimented throughout the site that is clearly related to the content (how it should be I guess)... Any comments, suggestions welcome
Intermediate & Advanced SEO | | CraigAddyman0