How to find all 301 redirect for URL xyz.com/products (internal and external)?
-
This is what we are thinking:
- Get all URL of the xyz.com/products using XENU software.
- Search those URL on google (site;xyz.com url ) to find out if they are crawled by google, do the same on bing (as currently google shows 4k URL and bing 11k )
- Use opensiteexplorer (301 redirect ) and using (internal external) to get the desired result.
Is this the right approach? If not, what is the best way to find the correct result?
All suggestions are welcome.
-
You are welcome to work with the URLs in search results. I am unsure what numbers you are attempting to match.
-
Thanks @Ryan
I was referring https://www.google.com/#q=site:domain.com using site:domain.com , I know it doesnt give me all urls but isnt what we care about, at least matching numbers I mean?
-
You can check the redirects by uploading the LIST of URLs to Screaming Frog, which will then crawl the list and inform you of any header responses (301, 302, etc)
I am unclear on your second question. You previously stated the site involved is not a client and you do not have access to Webmaster Tools. What exactly are you asking or suggesting?
-
Thanks Ryan and everyone else, amazing answers So here is my understanding:
For Internal 301:
- Use Xenu or Screaming to scan url and create the list. I hope we can get a clean report from screaming.
For External:
-
Use Ose to find all backlinks and save them on excel
-
Use AHREFs to find all backlinks and save them
-
Use Majestic to find all backlinks and save them
-
Combine all url and remove duplicate(I guess manually we gotta do that)
Questions
a) How do we find out which one is 301 redirected beside checking each of them?
b) for backlinks we should check what google and bing crawled?
-
I agree - screaming frog is an awesome tool for finding the internal 301s. You can export a spreadsheet of all the pages that contain the 301s etc and so it makes creating a task list to work through (or pass on) pretty straight forward.
-
Your original question expressed a desire to locate "all" redirects for a given URL. It is highly unlikely to locate all such links without access to the link data in Google WMT and Bing WMT, along with the referrer data in GA.
The best you can do to find external URLs is to build a comprehensive backlink report using data from multiple providers (OSE, AHREFs, Majestic, etc). You should know from the start you will not cover all the links unless you are working with pages which have a low number of links.
-
Hi Ryan,
Thanks for your quick response.
Yes we heard about the screaming frog but never used it, will give it a try.
We do not have access to google/bing webmaster tool or analytics. We are more like a third party company working on this project.
Any other ideas?
-
I used to use Xenu until shortly after Dr Pete shared this blog (http://moz.com/blog/crawler-faceoff-xenu-vs-screaming-frog) which introduced me to Screaming Frog. Both will work, but you will find XENU is more like using DOS whereas Screaming Frog has a very nice interface.
For internal 301s, you can clearly crawl the site and export a list of all 301 redirects to the target page.
For external URLs, there is not a simple method. I suggest two tactics:
1. Examine your analytics for referring URLs
2. Examine your backlink reports for links to the page
You can then crawl the list of URLs and determine which pages are being redirected. With the above understood, the primary concern should be your internal URLs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can building quality links on internal pages help us to improve DA?
Dear expert,
Moz Bar | | jhfdbeaob
I hope that you all are fine 🙂
Today, I am very confused about improving DA of my client website. For example: He has a domain www.abc.com
and he wants me to build a backlinks on different internal pages like
www.abc.com/service-1
www.abc.com/service-2
www.abc.com/service-3 If i build high quality links on internal pages (service pages) rather than building links on main domain (home page). Does the Domain Authority will increase
OR
I have to build links on Main page to boost the DA I am hoping a positive reply from your side Thanks1 -
Moz Bar not providing any data. Tried logging out/back in and un/re-installing, but no dice.
Used Mozbar for a long time, and normally works fine. Suddenly finding that it is not providing any data. All of the fields are there, but it does not provide me with PA/DA, etc, and all social metrics are at 0. This is across all sites, not just on in particular. Have tried logging out and in, deactivating and activating, and reinstalling. Nothing has worked.
Moz Bar | | SearchPros2 -
Moz is finding phantom pages
I suddenly have 4xx errors in my crawl diagnostics because pages with “/%3C/div” added to the end of the URL that are linked from the normal page can't be found. I didn't create the pages, and they don't exist, but Moz thinks that they do. I went back through to see if any changes in WordPress, theme or plugins versions might be the cause, but this is the only site that I have this issue, so I don't think that is it. Does anyone have an idea what causes this?
Moz Bar | | samuelldrew0 -
Why is the exact same URL being seen as duplicate and showing an error in my SEO reports
Well, I am still having duplicate page issues. I have a question about one of the errors SEO is giving me when I download a crawl report. I am going to attach a screen shot of part of the report so you can see for yourself, along with explaining it here. SEO shows the list of URL's that it crawled in the report. In this(see attachment) portion of the report it has 321 results for the exact same URL. It also says all of these exact same URL's have received a 404 error. What I want to know is how does it make 321 results for the same URL? And with this error that I don't see when I look at the page? 0hkRDST
Moz Bar | | JoshMaxAmps0 -
URL not returning a page successfully
I am trying to use the on page grader however it is not working with my website. The url is: http://www.britishhardwoods.co.uk/ Can anyone help?
Moz Bar | | British_Hardwoods0 -
URL inaccessible for On Page Grader
I am trying to use the on page grader however it is not working with my website. The URL is as follows: https://capbeast.com. I have been trying to read in older posts to see if https is now supported or not but have not found anything. I know there is no robots.txt issue as I am able to run the crawl test on our website fine. Is the issue on my end in regards to configuration or is due to DDos attacks? Any help would be appreciated. Thanks
Moz Bar | | MisterStitches0 -
Why can't On-Page Grader grade any Hilton hotel URLs?
I'm receiving the "Sorry, but that URL is inaccessible." for every hilton hotel webpage I check when using On-Page Grader. Is Hilton blocking Moz's On-Page Grader or is something else going on? Here are a few "inaccessible URLs" from different brands within Hilton's portfolio: http://doubletree3.hilton.com/en/hotels/new-york/doubletree-by-hilton-hotel-metropolitan-new-york-city-NYCDTDT/index.html http://home2suites3.hilton.com/en/hotels/tennessee/home2-suites-by-hilton-nashville-vanderbilt-tn-BNAHTHT/index.html http://hamptoninn3.hilton.com/en/hotels/florida/hampton-inn-and-suites-destin-DSINEHX/index.html http://hiltongardeninn3.hilton.com/en/hotels/georgia/hilton-garden-inn-atlanta-downtown-ATLDOGI/index.html Thanks in advance.
Moz Bar | | Just-Me0 -
Moz Crawl Test says pages have no internal links
Greetings, I am working on a website, https://www.nasscoinc.com, and ran a Moz Crawl Test on it. According to the crawl test, only 2 of the website's hundreds of pages are receiving internal links. When I run a similar test on the site using Screaming Frog, I see that most of the pages have at least one internal link. I'm wondering if anyone has seen this before with the crawl test; and there is a way to get the crawl test to see the internal links? Thanks!
Moz Bar | | TopFloor0