Technical Question about tools available in market
-
Hi,
I am looking for a tool most probably web based tool like opensiteexplorer / majestic seo that gives me the list of URL
For example, on google we can do site:seomoz.org , and it's saying About 115,000 results.
I need to get list of those 115,000 URLS in any file whether it's csv or any other.
anybody care to share ?
-
SEOmoz use to have this option in the past days but recently when i checked its gone... but while research to answer this question i found this http://www.martinhurford.com/serp-downloader-bookmarklet.html
Check this out and see if it is powerful!
-
I agree that Screaming Frog is the best. http://www.screamingfrog.co.uk/seo-spider/
You can get a full list of URLs including data on the following:
- Errors – Client & server errors (No responses, 4XX, 5XX)
- Redirects – (3XX, permanent or temporary)
- External Links – All followed links and their subsequent status codes
- URI Issues – Non ASCII characters, underscores, uppercase characters, dynamic uris, long over 115 characters
- Duplicate Pages – Hash value / MD5checksums lookup for pages with duplicate content
- Page Title – Missing, duplicate, over 70 characters, same as h1, multiple
- Meta Description – Missing, duplicate, over 156 characters, multiple
- Meta Keywords – Mainly for reference as it’s only (barely) used by Yahoo.
- H1 – Missing, duplicate, over 70 characters, multiple
- H2 – Missing, duplicate, over 70 characters, multiple
- Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, noodp, noydir etc
- Meta Refresh – Including target page and time delay
- Canonical link element & canonical HTTP headers
- X-Robots-Tag
- File Size
- Page Depth Level
- Inlinks – All pages linking to a URI
- Outlinks – All pages a URI links out to
- Anchor Text – All link text. Alt text from images with links
- Follow & Nofollow – At link level (true/false)
- Images – All URIs with the image link & all images from a given page. Images over 100kb, missing alt text, alt text over 100 characters
Thanks
-
ScreamingFrog the best
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Links being reported in Webmaster Tools
Hi Are the Total Links To Your Site, as reported in GWT, purely external inbound links ? Since these links are usually, as far as i can tell, much higher in number than any other link reporting tool and hence, i presume, more accurate, why don't services such as Moz etc include this in reporting ? I know its just a total number and link quality is whats important not quantity, but i would have thought interesting to show in reporting in conjunction with link quality info such as is already reported. Since most backlink reporting tools do show a total but always much much lower than that reported in gwt (i think) All Best Dan
Moz Pro | | Dan-Lawrence0 -
Any tool built into MOZ that can help tell who the owner of a URL is?
I'd like to know if there's any tool which would let us know who the owner of a web domain is.
Moz Pro | | daleseppie0 -
Mac Alternatives for Netpeak & SEO Tools For Excel?
Does anybody know of any mac alternatives for Netpeak & SEO tools for excel? I haven't been able to find any. I just need something to pull PA & DA quickly for a list of domains and URLs. Will I just have to create something custom with the Moz API?
Moz Pro | | kking41200 -
Google Webmaster Tools and Open Site explorer's links not matching up
My question is why do my GWT's links not match up to the ones on Open Site Explorer. I watched John Mueller's video, and he said that they had problems with the link counts recently. After checking my links today, I can see that the problem is fixed, but my link count differs from Open Site Explorer. On Web Master Tools I have 307 links, but on Open Site Explore I have 42. Has anyone dealt with this problem? thanks. Peter
Moz Pro | | PeterRota0 -
Local SEO Rankings Report Tools (based on several locations)?
Hi, Just a quick question. I am looking for a Rankings Report tool that allows a user to not only provide a list of Keywords, but also a list of Locations for which to check Google UK's ranking results. In practice, I would like to be able to enter 6 names of Towns next to my keyword list and get "localised results". Many thanks in advance, Cheers Greg
Moz Pro | | GregoryTK
PS: first Q within the Q&A, so please apologise for any potential "stupid questions"1 -
Keyword Difficulty Tool
Is there a way to use KDT and include my own URL in the process so that I can see (and show my client) how things look competitively across all these nice dimensions? All is well if my client's site is in the top 10 - but if it isn't, how can I get the same set of metrics on a specific URL as it pertains to a specific keyword? Do I somehow to remember it used to do this? Or am I imagining things? I can't seem to get it to work this way. Thanks,
Moz Pro | | seo_plus0 -
SEOMoz Campaign Tool
I've noticed that when looking at the SEOmoz tool, specifically the On Page analysis tool, it is still looking at an old url. About two months ago I made updates to all of our category page URLs. Previously the old urls were stuffed with keywords, strange characters and were really long. When looking at the on-page tool though it is referencing the old urls for keywords and I'm wondering why? I figure its been long enough to recognize the new urls. Is the paring of a keyword and a url saved and just graded on a weekly basis to produce the report? I had expected to see the new url's by now which are also represented in the sitemap. Around that same time I also added our TellAFriend Page and Review pages to our Robots.txt file as not to be crawled but I still see these pages come up in the errors report. Should this update as well?
Moz Pro | | dgmiles0 -
Any tools for scraping blogroll URLs from sites?
This question is entirely in the whitehat realm... Let's say you've encountered a great blog - with a strong blogroll of 40 sites. The 40-site blogroll is interesting to you for any number of reasons, from link building targets to simply subscribing in your feedreader. Right now, it's tedious to extract the URLs from the site. There are some "save all links" tools, but they are also messy. Are there any good tools that will a) allow you to grab the blogroll (only) of any site into a list of URLs (yeah, ok, it might not be perfect since some sites call it "sites I like" etc.) b) same, but export as OPML so you can subscribe. Thanks! Scott
Moz Pro | | scottclark0