Technical Question about tools available in market
-
Hi,
I am looking for a tool most probably web based tool like opensiteexplorer / majestic seo that gives me the list of URL
For example, on google we can do site:seomoz.org , and it's saying About 115,000 results.
I need to get list of those 115,000 URLS in any file whether it's csv or any other.
anybody care to share ?
-
SEOmoz use to have this option in the past days but recently when i checked its gone... but while research to answer this question i found this http://www.martinhurford.com/serp-downloader-bookmarklet.html
Check this out and see if it is powerful!
-
I agree that Screaming Frog is the best. http://www.screamingfrog.co.uk/seo-spider/
You can get a full list of URLs including data on the following:
- Errors – Client & server errors (No responses, 4XX, 5XX)
- Redirects – (3XX, permanent or temporary)
- External Links – All followed links and their subsequent status codes
- URI Issues – Non ASCII characters, underscores, uppercase characters, dynamic uris, long over 115 characters
- Duplicate Pages – Hash value / MD5checksums lookup for pages with duplicate content
- Page Title – Missing, duplicate, over 70 characters, same as h1, multiple
- Meta Description – Missing, duplicate, over 156 characters, multiple
- Meta Keywords – Mainly for reference as it’s only (barely) used by Yahoo.
- H1 – Missing, duplicate, over 70 characters, multiple
- H2 – Missing, duplicate, over 70 characters, multiple
- Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, noodp, noydir etc
- Meta Refresh – Including target page and time delay
- Canonical link element & canonical HTTP headers
- X-Robots-Tag
- File Size
- Page Depth Level
- Inlinks – All pages linking to a URI
- Outlinks – All pages a URI links out to
- Anchor Text – All link text. Alt text from images with links
- Follow & Nofollow – At link level (true/false)
- Images – All URIs with the image link & all images from a given page. Images over 100kb, missing alt text, alt text over 100 characters
Thanks
-
ScreamingFrog the best
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content: Marketing Page / Content Page
So I am getting duplicate content warnings on my website for my pages white paper and webinar video pages. Each white paper / webinar video page is behind a marketing form page that must be filled out. I am getting a lot of warnings that the marketing page and the content page are being picked up as duplicated content. In the past, both the marketing page and the content page were given the same title and url, the body content is not similar. My question: Is the URL / Title similarity enough to set off the duplicate content warnings and would changing one or the other solve the issue?
Moz Pro | | AllMedSeo0 -
A question on keywords that rank 51+
Good afternoon everyone. I wanted to pose a question to the group about keywords and the "on-page optimization - grade a page tool." I have a list of keywords that I am trying to rank for. Some of them are not ranked in the top 50, so on the keyword ranking tool it gives you the 51+ message in the rank column. For the items that are ranked I can try to improve them by looking at grade a page and typing in the URL and keyword. It will then give me a score and suggestions on how to improve it. With that being said, is there an easy way to find out which pages I should be optimizing those keywords which rank at 51+ for, besides typing the keywords in Google and seeing what URL it associates with the specific keyword? I hope the question above is clear.
Moz Pro | | trumpfinc0 -
Does SEOmoz have a Keyword Research tool similar to, say, the Google AdWords tool or the WebCEO Keyword Research Tool? And where might that be? (Sorry, I'm very new to SEOmoz Pro.)
I'm looking for an SEOmoz version of the classic WebCEO Keyword Research that would give you effective suggestions based on a keyword inquiry. I've made the switch from WebCEO, but I'm trying to find something similar to that Keyword Research tool. Am I going to just need to use the Google AdWords tool for this function or does SEOmoz have it's own version?
Moz Pro | | SmokewagonKen0 -
Why are results different on Rank Tracker tool and New Rankings Weekly Reports?
Good Morning... Every week i get an email from SEO Moz that is showing me where the keywords i am tracking are currently in the SERPS (New Rankings and On-Page Reports)... Is there a reason that the current rank shown on these emails would be so different than what is shown when i use SEO Moz's rank tracker? For example, it shows one keyword has a rank of 48 (via email) and a rank of 16 through the rank tracker. What would cause such a great discrepancy? Thanks
Moz Pro | | Prime850 -
Keyword Difficulty Tool not working!
Hello everyone, I quite new to the pro service and I am still on my free trial and I have to say that it is perfect! However, I haven't been able to use the keyword difficulty tool for a couple of days as it keeps saying that there is a problem and that I should try again in 20 minutes. I tried several times and 9 out of 10 times I get this message. I have only managed to get a couple of keywords analyzed. Keep in mind that most of the times I am not using English terms but since it worked in the past I would think that this is not the problem. Any thoughts on how to fix this issue? Thanks in advance!
Moz Pro | | Somondo1 -
Can the link acqusition tool / web directory list be used for non-english language enquiries?
Hi there, We are currently in the trial period of seomoz pro and have been testing all of the availiable tools. We are very happy with OSE and the other professional tools however we have discovered a barrier in our research methods. 1, Does anyone know how can you use the link acqusition assistant for non-english enquiries? The suggested search queries that are automatically generated are "submit site" "submit url" etc, but this has almost no use for a non-english keyword. - Also are the results only displayed for google.com - is there no localization at all of search enquiries? 2, The list of directories provided by seomoz link: http://www.seomoz.org/directories are anglo-centric. What are the availiable options for non-english speaking directories / local directories NOT based in the United States? Any help that can be provided would be greatly appreciated!
Moz Pro | | deliveryhero0 -
Keyword tool results - broad match or exact?
Do the SEOmoz keyword rankings display results for exact match or broad match?
Moz Pro | | ClaytonKendall0 -
Looking for a tool that can pull OSE stats for a bulk amount of URLs
I know that people have developed inhouse tools with the OSE API that can analyze thousands of URLs and pull metrics like PA, inbound links, etc. I need to analyze about 80k URLs and sort them by authority and I was hoping that someone could point me to a tool that can do this or let me use their tool. I'm willing to pay for access to it. We could build it inhouse, I imagine that it would be pretty easy, but our IT resources are stretched too thin right now.
Moz Pro | | Business.com0