Technical Question about tools available in market
-
Hi,
I am looking for a tool most probably web based tool like opensiteexplorer / majestic seo that gives me the list of URL
For example, on google we can do site:seomoz.org , and it's saying About 115,000 results.
I need to get list of those 115,000 URLS in any file whether it's csv or any other.
anybody care to share ?
-
SEOmoz use to have this option in the past days but recently when i checked its gone... but while research to answer this question i found this http://www.martinhurford.com/serp-downloader-bookmarklet.html
Check this out and see if it is powerful!
-
I agree that Screaming Frog is the best. http://www.screamingfrog.co.uk/seo-spider/
You can get a full list of URLs including data on the following:
- Errors – Client & server errors (No responses, 4XX, 5XX)
- Redirects – (3XX, permanent or temporary)
- External Links – All followed links and their subsequent status codes
- URI Issues – Non ASCII characters, underscores, uppercase characters, dynamic uris, long over 115 characters
- Duplicate Pages – Hash value / MD5checksums lookup for pages with duplicate content
- Page Title – Missing, duplicate, over 70 characters, same as h1, multiple
- Meta Description – Missing, duplicate, over 156 characters, multiple
- Meta Keywords – Mainly for reference as it’s only (barely) used by Yahoo.
- H1 – Missing, duplicate, over 70 characters, multiple
- H2 – Missing, duplicate, over 70 characters, multiple
- Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, noodp, noydir etc
- Meta Refresh – Including target page and time delay
- Canonical link element & canonical HTTP headers
- X-Robots-Tag
- File Size
- Page Depth Level
- Inlinks – All pages linking to a URI
- Outlinks – All pages a URI links out to
- Anchor Text – All link text. Alt text from images with links
- Follow & Nofollow – At link level (true/false)
- Images – All URIs with the image link & all images from a given page. Images over 100kb, missing alt text, alt text over 100 characters
Thanks
-
ScreamingFrog the best
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is there a tool on Moz or out on the internet that does bad link checker
I'm still pretty new to this and I was wondering if there is a free software, one of Moz or free out on the the internet that allows you to check bad links. I've done a lot of link building with citations and directories that for my clients industry. I just don't want to add their website and profile to a bad/risky directory and it penalizes my clients. I've seen a few out there, but I need one that is respectable and reliable. Any suggestions? I found one called bad neighborhood http://www.bad-neighborhood.com/text-link-tool.htm. Thanks Again, Benny
Moz Pro | | ACann0 -
How can I increase the number of links downloaded with the Opensite Explorer Just-Discovered tool?
I recently just tried to download the list from the OpenSite Explorer Just-Discovered tool and it only downloaded 233 links. When I checked to see how many URLs were available it was over 4,000. It does seem as though it only downloaded today's links although I'm not quite sure about that. Is there a way to increase the number of links downloaded to match what is listed in Opensite Explorer? Thanks.
Moz Pro | | searchysearchy0 -
How can I tell what day I added my keywords to the keyword ranking tool?
I can't tell if a keyword is newly ranking or if it just started ranking because I added it to the system. I have added keywords piecemeal over several months and I can't remember which ones I added on which date. Can anyone help me with this?
Moz Pro | | unitedairlines0 -
I am experimenting with Raven Tools and have a question
Since I am an in-house SEO I'm a little slower to adopt some of the paid tools out there, but I am thinking of jumping in and opting for the paid version of Raven Tools. Can anyone who's currently using Raven Tools suggest how and where is the best place to export backlinks info for import into the Link Manager? I want something that will give me as much of the data as possible already filled in (i.e. PR, domain authority, anchor text, follow/no follow information etc.). I don't mind if I have to rearrange it, I just don't want to have to be inputting things like PR and anchor text one at a time by hand. I do have SEOMoz pro and I see that I can pull a CSV of backlinks and do two separate reports, one for followed and one for not followed links, etc. I'm just wondering if anyone has any additional suggestions. How about Ahrefs.com? Is anyone using the paid version of that tool in conjunction with Raven Tools for Link Management? Is there a better tool out there for Link Management than Raven Tools? (I just found this link in a Q & A thread for an open source SEO panel to try...http://www.seopanel.in/ - anyone using that one?) Thanks all!
Moz Pro | | danatanseo0 -
Tools which scan urls for social data
Hi can anyone recommend any tools out there, which can allow me to scan a list of pages (urls) and give me back social data for each page (e.g. number of facebook likes, shares, twitter data, google plus, etc) Cheers, Chris
Moz Pro | | monster990 -
Crawl test tool from SEOmoz - which URLs does it actually crawl?
I am using for the first time the crawl test tool from SEOmoz and I do not really understand which URLs the tool is going to crawl. First, it says "enter any subdomain" --> why can´t I do the crawl for the root domain? Second it says "we'll crawl up to 3,000 linked-to pages" --> does that mean that the tool crawls all internal links that it can find on the given domain? Thanks for your help!
Moz Pro | | Elke.GetApp0 -
Any ideas about using research tools data to create client interest in dedicating marketing spend on SEO services vs their other current antiquated marketing methods?
I consult in the dental industry. Many professionals in this industry are operating in the dark ages when it comes to marketing. Any ideas about using research tools data to create client interest in dedicating marketing spend on SEO services vs their other current antiquated marketing methods? I am approaching potential clients to pitch SEO. They aren't necessarily seeking SEO services currently but have a strong marketing budget. I would like to use seomoz tools data about their site and competitors as way to show possibilities/potential of SEO services to gain interest in further discussion of SEO benefits.
Moz Pro | | cody_baird0 -
SEOMoz Campaign Tool
I've noticed that when looking at the SEOmoz tool, specifically the On Page analysis tool, it is still looking at an old url. About two months ago I made updates to all of our category page URLs. Previously the old urls were stuffed with keywords, strange characters and were really long. When looking at the on-page tool though it is referencing the old urls for keywords and I'm wondering why? I figure its been long enough to recognize the new urls. Is the paring of a keyword and a url saved and just graded on a weekly basis to produce the report? I had expected to see the new url's by now which are also represented in the sitemap. Around that same time I also added our TellAFriend Page and Review pages to our Robots.txt file as not to be crawled but I still see these pages come up in the errors report. Should this update as well?
Moz Pro | | dgmiles0