Technical Question about tools available in market
-
Hi,
I am looking for a tool most probably web based tool like opensiteexplorer / majestic seo that gives me the list of URL
For example, on google we can do site:seomoz.org , and it's saying About 115,000 results.
I need to get list of those 115,000 URLS in any file whether it's csv or any other.
anybody care to share ?
-
SEOmoz use to have this option in the past days but recently when i checked its gone... but while research to answer this question i found this http://www.martinhurford.com/serp-downloader-bookmarklet.html
Check this out and see if it is powerful!
-
I agree that Screaming Frog is the best. http://www.screamingfrog.co.uk/seo-spider/
You can get a full list of URLs including data on the following:
- Errors – Client & server errors (No responses, 4XX, 5XX)
- Redirects – (3XX, permanent or temporary)
- External Links – All followed links and their subsequent status codes
- URI Issues – Non ASCII characters, underscores, uppercase characters, dynamic uris, long over 115 characters
- Duplicate Pages – Hash value / MD5checksums lookup for pages with duplicate content
- Page Title – Missing, duplicate, over 70 characters, same as h1, multiple
- Meta Description – Missing, duplicate, over 156 characters, multiple
- Meta Keywords – Mainly for reference as it’s only (barely) used by Yahoo.
- H1 – Missing, duplicate, over 70 characters, multiple
- H2 – Missing, duplicate, over 70 characters, multiple
- Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, noodp, noydir etc
- Meta Refresh – Including target page and time delay
- Canonical link element & canonical HTTP headers
- X-Robots-Tag
- File Size
- Page Depth Level
- Inlinks – All pages linking to a URI
- Outlinks – All pages a URI links out to
- Anchor Text – All link text. Alt text from images with links
- Follow & Nofollow – At link level (true/false)
- Images – All URIs with the image link & all images from a given page. Images over 100kb, missing alt text, alt text over 100 characters
Thanks
-
ScreamingFrog the best
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content report - question on best practice
Hello all, New to MOZ Pro and SEO - so lots to get my head round! I’m working through the Duplicate Content section of the Crawl report and am not sure what the best practice is for my situation. Background: We are a reference guide for luxury hotels around the world, but the hotels that are featured on the site vary year on year. When we add a new hotel page, it sets up the url as ourwebsite.com/continent/country/regionORcity/hotel. When the hotels come off, I redirect their URL to the country or region where we have other hotels. Example: http://www.johansens.com/europe/switzerland/zermatt/ The hotel in Zermatt has come off the site, showing 0 results on this landing page. Question: My duplicate content report is showing a number of these regional pages that are displaying the copy “0 places - Region’ because the hotel has come off, but the landing page is still live. Should I redirect the regional page back to the main country page? And then if I add a new hotel to the site from that region in the future, simply remove the redirect? Should I also delete the page? Any tips would be much appreciated!
Moz Pro | | CN_Johansens0 -
Webmaster Tools shows mystery errors that Moz does not
One of my campaigns is doing great in the sense that the website has been running fault free for a few months now. Great, of course! But... in Google Webmaster Tools errors keep coming in showing older media documents and pages. And it does not say where they are from. Probably this is more a Google question, but I thought I'd try to find some answers here first. I would appreciate any suggestions and help. Monique
Moz Pro | | MarketingEnergy0 -
Do I need to set the country in Webmaster Tools even if I am set to apply the hreflang tag to our different country pages?
We are global brand that will be migrating to a new platform in the next few months. As such, this will allow us to fix our current SEO issues. We are planning on using geo-targeted subfolders instead of country code top level domains primarily because of resource limitations. My question is - do I need to set the country in Webmaster Tools even though I am already going to tell Google that certain pages are targeted towards a specific country?
Moz Pro | | marshseo0 -
Internal and external links showing in Webmaster Tools but not in MOZ
Hi there I set up a campaign 2 weeks ago but I am registering zero internal or external links for my website. It's a small site with only about 300 external links but they show in Webmaster tools but not in my link report or Open Site Explorer. Do I need to wait until the next OSE crawl or is something wrong with my set up? Thanks in advance! Laurie
Moz Pro | | turnbullholdingsltd0 -
Update in Moz spider/tools?? Flagging duplicate content / ignoring canonical
Hi all, Has there been an update in the SEOmoz crawling software? We now have thousands of dupe content/page title warnings for paginated product page URLs that have correctly formatted canonicals. e.g. http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx ... has following pages with identical content that have been flagged: http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx?p=true&rspage=4 http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx?p=true&rspage=6 http://www.woolovers.com/british-wool/mens/olive-green/wool-countryman-suede-patch-sweater.aspx?p=true&rspage=4 ..plus 4 more URL's. But they all have canonical set. There's even a notice at the bottom of report that tells us there's a canonical set to http://www.woolovers.com/british-wool/mens/tweed-green/wool-countryman-suede-patch-sweater.aspx What gives, SEOmoz ?? Thanks Michael
Moz Pro | | LawrenceNeal0 -
Page authority questions?
I've been analyzing some IT communities ...in order to check how relevant is the page authority vs PageRank. I found one main site which is organized by "communities'..and every community is a sub-domain. The root domain has an authority of 90/100 which it should be great......so the sub-domains "inherit" part of this authority.... Until here everything seems to be perfect. However, I went deeper and I picked one of these communities. Analyzing the "Linking Root Domain" I discovered it only has only 5 root domains pointing to its home page. Those 5 Root Domains have generated more than 134k links. That doesn't seem to be "natural". Checking those 5 Root Domains I discovered that they have been registered by the same Root Domain site. Ex: Main domain: Domain.com Community1.domain.com Community2.domain.com.... Linking Root Domains: DomainXY.com DomainABC.com DomainRST.com DomainFGH.com DomainOPQ.com It seems to me that it is easy to cheat the authority domain score. Just creating others sites developing the same topic and generating back links to your main domain
Moz Pro | | SherWeb0 -
Question about difficulty score of keywords
I have a question regarding keyword selection and the difficulty score. I get the idea that you need to temper your desire to go after keyword phrases that generate lots of traffic with the reality of competition for that keyword, which is conveyed by the difficulty score. But how do we know when the difficulty score is too high? And isn't this relative to a website? If my website is in the top 1% of traffic, can't I go after more difficult keywords than if my traffic is in the top 10%? Are there guidelines or best practices to help with this decision? How do I know whether my website is "strong" enough to compete for a keyword that has a high difficulty score. If we already rank on page one of Google for numerous keywords with difficulty scores of 50 or so, is that an indicator that we can try to rank for other keywords with a difficulty score of 50? When do we know that we can start trying to rank for more diffcult keywords?
Moz Pro | | espressob2b0