How to visualize our entire site to discover the origin of URLs?
-
What is a tool to use so that I can visualize all links to all pages on the site so that I can discover how certain duplicate content URLs are being created?
-
SEO Spider from Screaming Frog is a great crawler and a notable alternative to Xenu
-
I have not used the tool in quite some time but if I recall correctly it does offer a referring page URL. I can't imagine it would be useful without that information.
-
Yea, I've seen that before. Will that show me how unwanted pages are being referenced?
-
I believe this tool will meet your needs.
-
What if I need 150k pages crawled, rather than 20k?
-
If you set up a SEOmoz campaign, the web crawl report will offer that information. If not, you can request your site to be crawled: http://pro.seomoz.org/tools/crawl-test
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL Question: Is there any value for ecomm sites in having a reverse "breadcrumb" in the URL?
Wondering if there is any value for e-comm sites to feature a reverse breadcrumb like structure in the URL? For example: Example: https://www.grainger.com/category/anchor-bolts/anchors/fasteners/ecatalog/N-8j5?ssf=3&ssf=3 where we have a reverse categorization happening? with /level2-sub-cat/level1-sub-cat/category in the reverse order as to the actual location on the site. Category: Fasteners
Technical SEO | | ROI_DNA
Sub-Cat (level 1): Anchors
Sub-Cat (level 2): Anchor Bolts0 -
URL Structure
Hi, Hope you are all well. On our website we have a 'blog' and a 'news' section. The blog is located on "/blog" - but when you click on a post the url structure changes to /name-of-article and the blog subdomain isn't included. Would it be better to have "blog/name-of-article as this would then make the blog perform better in search results? Also, if our news page is under /news - but when you click on an article it changes to /news-article/name-of-article Wouldn't it be better to have /news/name-of-article Thanks a lot!! 🙂
Technical SEO | | National-Homebuyers0 -
301 for a Very Long URL
Hey gang, Thanks ahead of time for the help. I have a url somehow that is very very long: http://www.colbysphotography.com/wedding-caterers-knoxville-east-tennessee/Here is an extensive list of wedding venues in the Knoxville and East Tennessee region. If you find that any of these links are not working, that the venues are no longer in business, or have a suggestion for an additional venue (at no charge), please contact Colby. Colby's Photography works hard on keeping this list helpful. I have tried Yoast Premium on a wordpress site to redirect the url but it doesn't seem to keep. I've tried a few other redirect plugins with not help either. I would love some suggestions on this one! Colby
Technical SEO | | littlecolby0 -
What directory should a site go in (url structure)?
Hi All, The is the first actual SEO campaign i've worked on and I had a few question about where the site should live on the server and url structure. The site is in WP and we're using Yoast SEO. Anyway the site lives in a a folder called Coastal, which is a child of the WWW folder. So the permalink of the homepage is mcoastalwindows.com/coastal/. The URL is mycoastalwindows.com. The thing is I can still get to the homepage or any of the pages on the site by typing in the /coastal/. Another example is permalink mycoastalwndows.com/coastal/siding/ and url mycoastalwindows.com/siding/. The urls always display without the /coastal/, so I'm not too worried about people linking to them, but Yoast puts a canonical element to the permalink and always includes the /coastal/. Also I'm seeing that Google displays a lot of the urls with the /coastal/, which is an issue seeing as we don't link to the pages that way. My original thought was to solve this at the source and just move everything out of the coastal directory, but the developer swears that it's more secure being in another folder especially with WP. What would you all do and what is best practice? Would you move everything out of the coastal folder, 301 re-direct, do something with. htaccess, or another solution? Appreciate the input thanks!
Technical SEO | | Mario.Souza0 -
Internal Ads on A Site
We serve ads on our site using a sub-domain. All ads use a re-direct from ads.domain before redirecting users to the proper, normal, internal url. Most the content on our home page is ad block driven. Is it possible and does it make sense to enter the sub-domain as url parameter in Google Webmaster tools, letting Google know that this is something to be ignored. Many thanks
Technical SEO | | CeeC-Blogger0 -
Is 301 redirecting all old URLS after a new site redesign to the root domain bad for SEO?
After a new site redesign ...would it hinder our rankings if we 301 redirected all old URLS that are returning 404 error codes to the root domain (home page) ? Would this be a good temporary solution until we are able to redirect the pages to the appropriate corresponding page? Thanks so much!
Technical SEO | | DCochrane0 -
Help optimising this site
Hi I have been optimising this site http://seakayakdevon.co.uk/ which is a wordpress site since making changes to it recently the site is now indexed and appearing among its competitors. trouble is they still are placed higher rn the SE rankings. i wish to optimise for local search i.e on Google places etc. but the trouble is there is'nt a physical address for the business it is run from various coastal locations. any ideas how i can still market for local search- maps etc. I have done the following optimisation: sitemaps title tag, description tag improved content removed duplicate content an blocker pages replace image text and replaced with header tag improved page names - making them static any advice of guidance would be greatly appreciated- will the fatc its built in wordpress limit its ability to gain better ranking in the SE? Thanks
Technical SEO | | Bristolweb0 -
URL Rewrite
We are trying to convince a client to do a massive rewrite from all URL's looking like this: "www.company.com/category/categoryId=82374" to something like "www.company.com/womens/jackets/rain" How would you describe the importance and impact of doing URL rewrites to an ecommerce site? What evidence/research can we share with them to convince them it is worth the time and effort to do?
Technical SEO | | Hakkasan0