How to find all 301 redirect for URL xyz.com/products (internal and external)?
-
This is what we are thinking:
- Get all URL of the xyz.com/products using XENU software.
- Search those URL on google (site;xyz.com url ) to find out if they are crawled by google, do the same on bing (as currently google shows 4k URL and bing 11k )
- Use opensiteexplorer (301 redirect ) and using (internal external) to get the desired result.
Is this the right approach? If not, what is the best way to find the correct result?
All suggestions are welcome.
-
You are welcome to work with the URLs in search results. I am unsure what numbers you are attempting to match.
-
Thanks @Ryan
I was referring https://www.google.com/#q=site:domain.com using site:domain.com , I know it doesnt give me all urls but isnt what we care about, at least matching numbers I mean?
-
You can check the redirects by uploading the LIST of URLs to Screaming Frog, which will then crawl the list and inform you of any header responses (301, 302, etc)
I am unclear on your second question. You previously stated the site involved is not a client and you do not have access to Webmaster Tools. What exactly are you asking or suggesting?
-
Thanks Ryan and everyone else, amazing answers So here is my understanding:
For Internal 301:
- Use Xenu or Screaming to scan url and create the list. I hope we can get a clean report from screaming.
For External:
-
Use Ose to find all backlinks and save them on excel
-
Use AHREFs to find all backlinks and save them
-
Use Majestic to find all backlinks and save them
-
Combine all url and remove duplicate(I guess manually we gotta do that)
Questions
a) How do we find out which one is 301 redirected beside checking each of them?
b) for backlinks we should check what google and bing crawled?
-
I agree - screaming frog is an awesome tool for finding the internal 301s. You can export a spreadsheet of all the pages that contain the 301s etc and so it makes creating a task list to work through (or pass on) pretty straight forward.
-
Your original question expressed a desire to locate "all" redirects for a given URL. It is highly unlikely to locate all such links without access to the link data in Google WMT and Bing WMT, along with the referrer data in GA.
The best you can do to find external URLs is to build a comprehensive backlink report using data from multiple providers (OSE, AHREFs, Majestic, etc). You should know from the start you will not cover all the links unless you are working with pages which have a low number of links.
-
Hi Ryan,
Thanks for your quick response.
Yes we heard about the screaming frog but never used it, will give it a try.
We do not have access to google/bing webmaster tool or analytics. We are more like a third party company working on this project.
Any other ideas?
-
I used to use Xenu until shortly after Dr Pete shared this blog (http://moz.com/blog/crawler-faceoff-xenu-vs-screaming-frog) which introduced me to Screaming Frog. Both will work, but you will find XENU is more like using DOS whereas Screaming Frog has a very nice interface.
For internal 301s, you can clearly crawl the site and export a list of all 301 redirects to the target page.
For external URLs, there is not a simple method. I suggest two tactics:
1. Examine your analytics for referring URLs
2. Examine your backlink reports for links to the page
You can then crawl the list of URLs and determine which pages are being redirected. With the above understood, the primary concern should be your internal URLs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why are Internal, nofollowed links good for Domain Authority?
I always thought that internal links should be 'followed' to preserve link-juice.
Moz Bar | | BottleGreenWebsites
I've just been checking my Domain Authority Links overview, and the competitor with the most no-follow internal links has been rewarded with a higher Moz DA score.
Can anyone explain why this would be the case? JRJ4Dwn1 -
What does it mean when an inbound external link has no anchor text?
There are 11k inbound external links on a client's website what have are "follow" links and apperently there is no anchor text for the link.
Moz Bar | | Harley.Helmer0 -
GWT and internal link emulation
Hi, I have a gwt site - www.whatiswhere.com. I have tab control which emulates the menue. I am planning to put <a>links instead of text into tab labels to create internal links. I am thinking to add java script to onclick event of</a> <a>otherwise i will get to the new session of GWT site. What I want is to just change the tab but at the same time have a link for the crawler. Would my approach work? Will it be equivalent to non-follow link? Will it improve the ranking comparing to 'no link at all' case?</a> <a>Thanks, Andrei.</a>
Moz Bar | | Anazar_20010 -
In the Moz Site Crawl, what does "External Links" mean?
I thought I knew what it meant but am finding instances where the value in the column, "Linking Root Domains" is greater than the value in the column, "External Links?" Thanks!
Moz Bar | | Edward_Sturm1 -
External links that don't exist
I did a crawl test and it shows that there are 2215 external links on my home page here: https://www.playshakespeare.com An "External Link" is defined here: https://moz.com/researchtools/crawl-test "Number of links on a page that point to a page that is not on the same domain as that page." There are NOT 2215 external links on that page. There are only a handful at any given time. Why is the crawl test reporting thousands over that?
Moz Bar | | playshakespeare0 -
What does the Bold/ Strong mean in Moz bar?
Under On-Page Elements in the Moz bar there is a Tag/ Location called Bold/ Strong. What does that mean?
Moz Bar | | TiffanyatElite0 -
Moz crawler finding my homepage multiple times
Hi and thank you in advance for your help! I have a Moz Pro campaign running (I am a complete Moz novice by the way) for one of my websites (balloonsutah.com). After crawling my site, the Moz crawler informed me that I have 3 pages with duplicate content. While I am not sure why exactly this is happening, the crawler indexed my homepage 3 times under different url's. -balloonsutah.com
Moz Bar | | Keenan-Price
-balloonsutah.com/
-balloonsutah.com/index.html I checked my FTP server and I cannot figure out for the life of me why the crawler is finding anything other than the index.html file. I suppose I need to do something regarding a rel="Canonical" but I am not terribly familiar with that either. Any suggestions would be greatly appreciated!
Keenan0