Software/tool for Content Inventory
-
Is there any tool out there which can crawl a section of a website (or whole website) and gives a list of urls by subsections?
I have tried PowerMapper but it only gives visual map, not a list.
Thanks!
-
You can use Moz, Screaming Frog, Sort Site to get results.
-
Xenu mya work http://home.snafu.de/tilman/xenulink.html
-
How about Screaming Frog? - http://www.screamingfrog.co.uk/seo-spider/
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
News site and duplicate/cannibalism/old
Hey everybody. I am working with a handful of medical news websites and some of these sites are rather large. I am noticing/suspecting there is some cannibalism going on here. How do news sites deal with this? Do they noindex the old stuff? Update the old stuff instead of writing a new piece all together? Curious what to do!
On-Page Optimization | | HashtagHustler0 -
Canonical: Same content but different countries
I'm building a website that has content made for specific countries. The url format is: MyWebsite.com/<country name="">/</country> Some of the pages for <specific url="">are the same for different countries, the <specific url="">would be the same as well. The only difference would be the <country name="">.</country></specific></specific> How do I deal with canonical issues to avoid Google thinking I'm presenting the same content?
On-Page Optimization | | newbyguy0 -
Horrible ranking for hundreds of content pages
We have hundreds of pages of restaurant reviews (each is 100% original content) on our site (for example, here is a list of many of the content pages: https://www.rewards21.com/articles). The PA of almost every page is 1 (ouch). Doing even the most long-tail Google search doesn't bring up one of those pages (even if I include our company name in the search string). I'm certain we have lots of room to improve SEO, but not even seeing the pages come up for even the most long-tail search has me concerned... Our site used to be on Wordpress many months ago and many of these pages ranked fine..ever since switching away from Wordpress, we've not made any SEO progress. Am wondering if something major is "broken" with our site SEO...
On-Page Optimization | | r210 -
Does a / at the end of a URL create a duplicate page?
Hello, I have just used (the amazing) Screaming Frog to check my site and it is listing the two following pages as having duplicate titles, making me think it is seeing them as duplicate pages. http://zenplugs.com/zenplugs-molded-earphones/ http://zenplugs.com/zenplugs-molded-earphones Do I need to redirect one of these? Thanks in advance! Toby
On-Page Optimization | | T0BY0 -
Duplicate Content: Snippets from Blog on Website
Our company has a website and a blog, each on a separate domain. On our website's home page, we include snippets from some of our blog posts and links to such posts. The snippets include the same photo, title, and first line or so of text. We have been told and am concerned that search engines will categorize this as duplicate content. Any recommendation as to how we can get around this issue aside from not including the blog posts on our site? I imagine we are not the first company to want to do this. www.angelicolaw.com Thanks,
On-Page Optimization | | BrazilLaw
Greg0 -
What is the right schema.org link for a web design / developer / mobile agency?
It seems strange that a group of web developers would make up an entire structured language to designate businesses by category and somehow forget to include companies like.... web developers. So I must be missing it, what is correct to use?
On-Page Optimization | | yeagerd0 -
Templated content with minor changes
Hello, I am working on a site with 1000s of inventory pages. i am slowly building links and writing custom content for pages that will bring best ROI. Will i have issues on templated pages with minor changes.currently traffic is pretty good. thank you
On-Page Optimization | | TP_Marketing0 -
Duplicate content - what to do?
Hi, We have a whole lot of articles on our site. In total 5232 actually. The web crawler tells me that in the articles we have a lot of duplicate content. Which is sort of nonsense, since each article is unique. Ah, some might have some common paragraphs because they are recurring news about a weekly competition. But, an example: http://www.betxpert.com/artikler/bookmakere/brandvarme-ailton-snupper-topscorerprisen AND http://www.betxpert.com/artikler/bookmakere/opdaterede-odds-pa-sportschef-situationen-pa-vestegnen These are "duplicate content", however the two article texts are not the same. The menu, and the widgets are all the same, but highly relevant to the article. So what should I do? How can i rid myself of these errors? -Rasmus
On-Page Optimization | | rasmusbang0