Technical Question about tools available in market
-
Hi,
I am looking for a tool most probably web based tool like opensiteexplorer / majestic seo that gives me the list of URL
For example, on google we can do site:seomoz.org , and it's saying About 115,000 results.
I need to get list of those 115,000 URLS in any file whether it's csv or any other.
anybody care to share ?
-
SEOmoz use to have this option in the past days but recently when i checked its gone... but while research to answer this question i found this http://www.martinhurford.com/serp-downloader-bookmarklet.html
Check this out and see if it is powerful!
-
I agree that Screaming Frog is the best. http://www.screamingfrog.co.uk/seo-spider/
You can get a full list of URLs including data on the following:
- Errors – Client & server errors (No responses, 4XX, 5XX)
- Redirects – (3XX, permanent or temporary)
- External Links – All followed links and their subsequent status codes
- URI Issues – Non ASCII characters, underscores, uppercase characters, dynamic uris, long over 115 characters
- Duplicate Pages – Hash value / MD5checksums lookup for pages with duplicate content
- Page Title – Missing, duplicate, over 70 characters, same as h1, multiple
- Meta Description – Missing, duplicate, over 156 characters, multiple
- Meta Keywords – Mainly for reference as it’s only (barely) used by Yahoo.
- H1 – Missing, duplicate, over 70 characters, multiple
- H2 – Missing, duplicate, over 70 characters, multiple
- Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, noodp, noydir etc
- Meta Refresh – Including target page and time delay
- Canonical link element & canonical HTTP headers
- X-Robots-Tag
- File Size
- Page Depth Level
- Inlinks – All pages linking to a URI
- Outlinks – All pages a URI links out to
- Anchor Text – All link text. Alt text from images with links
- Follow & Nofollow – At link level (true/false)
- Images – All URIs with the image link & all images from a given page. Images over 100kb, missing alt text, alt text over 100 characters
Thanks
-
ScreamingFrog the best
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is there an external tool to check for Link Balance profile (anchor text)?
Hi, Regarding the topic of Link Balance (anchor text from external sites). Is there an external tool you can point to your site or an individual URL that can give you the link balance (branded terms, exact match etc). This article is great:
Moz Pro | | STL
http://moz.com/blog/anchor-text-distribution-avoiding-over-optimization But it's a lot of work to do for individual pages (of which we have many). Just trying to see if there's a tool out there (Moz or otherwise), that can report on such balance on a page-by-page basis (paste in URL, hit 'Go'). Thanks.
Rich Talbot0 -
Help me to know why are all pages not being tracked by the Moz tool for on-page optimization reports?
The On-page Optimization report that the Moz tool shows, is not tracking all the pages from my website. I know this because it isn't showing a ranking for all pages on my website. Is there a particular reason why this is happening? It is important for me to know details of all pages, else it does not give me a comprehensive picture of what's going on in SEO.
Moz Pro | | jslusser0 -
Open Site Explorer vs Webmaster Tools
Hi there. OSE is showing 53 linking domains and WMT is showing 161.
Moz Pro | | JeromeSavon
Why are so many missing from OSE. They are all links of a decent age. Thanks0 -
How to make Seo Tools Site serves ?
hello i want to add seo tools at my site for users i want my visitors able to use pagerank checker , link tracker , backlinks checker , etc - and keyword tools , domain tools , Analytics and Reporting. something like http://smallseotools.com/ what scripts i need ? can i do this all with wordpress ? Thanks 🙂
Moz Pro | | Wagdys0 -
Keyword rankings are not updating from past two weeks? Is there any issue in SEOmoz tool ?
I have added 215 keywords to my account. From past two weeks i am waiting to see the rankings, however it's not updating.
Moz Pro | | Sasank0 -
Social Media Marketing
Hello People, I hope everyone is well, this is bugging me because I cannot figure it out, my old website which I still have up ranks on page 2 or 3 for the keywords "social media marketing edinburgh" but when I check the URL's in particular the SMM page: http://www.fentonit.co.uk/index.php?option=com_content&view=article&id=3 and I know the URL's a mess but we have resolved all these issues with the new co. setup. Anyway the On page optimization for this page is an F for this phrase, yet it ranks highly, I am greatly puzzled as to why. Can some one please help shed some light on this? Thanks to everyones who may kindly take the time to assist here. Craig www.fenwaymedia.co.uk
Moz Pro | | craig_sepperro.com0 -
Anyone have a free tool to create a xml sitemap?
Or can I use the Custom Crawl tool to help create this? The domain I'm working with has 1,300+ pages so most free tools I've used in the past won't capture that many pages.
Moz Pro | | JonClark150 -
SEOMoz Campaign Tool
I've noticed that when looking at the SEOmoz tool, specifically the On Page analysis tool, it is still looking at an old url. About two months ago I made updates to all of our category page URLs. Previously the old urls were stuffed with keywords, strange characters and were really long. When looking at the on-page tool though it is referencing the old urls for keywords and I'm wondering why? I figure its been long enough to recognize the new urls. Is the paring of a keyword and a url saved and just graded on a weekly basis to produce the report? I had expected to see the new url's by now which are also represented in the sitemap. Around that same time I also added our TellAFriend Page and Review pages to our Robots.txt file as not to be crawled but I still see these pages come up in the errors report. Should this update as well?
Moz Pro | | dgmiles0