Does anybody know of a good bulk import http response checker? The one I was using has disappeared and I can only find checkers that only take one URL at a time.
-
An example of something I'm looking for but I want to check multiple checkers: http://web-sniffer.net/
-
screaming frog does not seem to export the whole chain though when there is more than one redirect in a chain. Did you experience the same thing, as it may be a glitch on my side?
-
Jeremy - it's available in the free version and if you click on Mode > List you can upload a file. I'm crawling 75K URLs and had to manually increase my memory so it would crawl faster.
Instructions on that here:
http://www.screamingfrog.co.uk/seo-spider/user-guide/general/#5
Good luck!
-
I looked at screaming frog, I didn't see anything that indicated how you would provide a file of thousands of URL's on the same domain to be checked? And I don't want to crawl the website, I just want it to check the 1 url and provide a response then go to the next one. Where would one find this feature in screaming frog?
-
As the thread discusses, a while back we started using Screaming Frog and love it. Recommended.
-
Jeremy/Kurt - I'm in the same boat. Have 75K URLs I'm trying to check as part of a link pruning project.
There's actually another Moz Forum post (http://moz.com/community/q/mass-404-checker) that has a very similar ask and responses are largely the same.
Below is a link that was mentioned in this post, but I've tried it a few times and it keeps getting hung up on any list over 100 URLs.
http://www.tomanthony.co.uk/tools/bulk-http-header-compare/
Below are the only other ones I've found, but again are limited to 100 or less URLs.
http://tools.seobook.com/server-header-checker/?page=bulk&typeProtocol=11&useragent=1
I feel like this is such a common problem for people trying to do any kind of mass back link research project. I'm also looking for something that checks follow vs nofollow so I can further narrow my list.
Maybe we should just create the tool? : )
-
Not sure if we are attempting to do the same thing but in looking at the tools recommended here not sure they do what we are looking for. We would like to take a list of 100,000 URL's and have some software check the response code for each one. We don't want it to crawl a site just the individual URL provided and then go on to the next one in the list.
Anyone know of a tool that can do this?
Thanks,
Jeremy
-
Hi Anthony - we found Screaming Frog a few weeks ago. Definitely a solid tool.
-
Check out Screaming Frog - It's an SEO spider that crawls your site pretty quickly - giving a multitude of useful seo data - including response headers. Two drawbacks: 1) it costs roughly $130 and 2) it has difficulty crawling sites with more than 100K pages. I believe Audette Media is coming out with an industrial strength crawler this summer but don't have a lot details yet. Hope this helps.
-
Hey,
The Crawl Test includes the http status code for each page if that's what you're after. Then you can download it as a .CSV
DD
EDIT - Didn't realise this was PRO but you could always do the free trial
-
so far no luck!
-
Did the SEObook tool in the link below solve your problem, or are you still looking for a tool?
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can Google shopping listings affect SEO rankings?
Hello everyone. Here is my question: Until 4 weeks ago we used to have just around 2,000 product/pages on Google shopping of our product pages, then we added over 40,000 more items/pages to it, and our Google natural search rankings dropped on most of our long-tail keywords pointing to the same product pages. Do you think that's just coincidence or could the number of products/pages on Google shopping affect natural search results?
Paid Search Marketing | | fablau0 -
Adwords Customizers - Possible to show different prices by country of visitor in adwords using adwords customizers?
Is it possible to show different prices by country of visitor in adwords using adwords customizers? I would like to avoid having to setup campaigns for each country, in order to show country specific prices in adwords. thanks
Paid Search Marketing | | lcourse0 -
Will User See More Than One of My Facebook Ads
I have numerous campaigns set up in my Facebook Advertising account. If I have different ads in different campaigns but with the same geographic and demographic targeting, could one individual see multiple instances of my advert?
Paid Search Marketing | | Superdream0 -
Can I dynamically add city name to my PPC ad text and URL based on the user's search?
I have looked into DKI (Dynamic Keyword Insertion), but have not found a solution and thought that some excellent Mozzer might be able to help. Here is the idea: We have landing pages for hundreds of cities. The local content on each of these cities changes page to page, however the keywords that we are going after are the same. So, I am trying to create a dynamic ad group that looks something like this: Headline: {City Name} {Keyword} Description: We cover {City Name} {Keyword}, get more info now! URL: http://www.website.com/{City Name} Please let me know if you can assist with this, B
Paid Search Marketing | | Reis_Inc.0 -
Adwords enhanced campaigns - Specify alternate destination URLs
Morning Mozzers, I am setting up a new campaign for a client, they would like to target mobile devices which evidently i can no longer do with enhanced campaigns; however i have increased the Mobile bid as much as possible and set the default CPC low to try and minimise appearance in desktop search. The client has a desktop and a mobile version of the destination page but the site will not direct users to the correct page based on their device as they are two separate domains. As such i want to know if i can specify an alternate destination URL for an ad in Adwords based on if the click comes from a mobile or desktop device? The other option is to set all my Ads within the Adgroup to have mobile as the device preference and just use the mobile landing page, but not sure if there is a neater solution here? Thanks, Tom.
Paid Search Marketing | | Sarbs0 -
Trademarked words in in Google Adwords ads - Why do competitors get to use them?
Hi, The keyword I want to use in my ad is trademarked, so they disqualified my ad. The trademark was specifically cited as the reasoning. I tried this across maybe 5 different ads. All disqualified The thing I don't understand is that there are like 10 other advertisers who are actively using this "trademarked" word in their ads. It's not like 1 scooted past Google, there's a ton of advertisers doing it. So how do I get past them or were they grandfathered in or something? FYI... I tried dynamic insert to see if that could my "trademarked" word in the back door, but no luck. Any other ideas? Thanks!
Paid Search Marketing | | marketingcupcake0 -
Can I pass keyword match type through the Adwords dest URL?
Hi guys, I'm currently using duplicates of the same keyword with different match types, and I'd like to pass the keyword match type through the query, the same way I can pass the keyword and creative. ie. http://my.site.com/?kw={keyword}&kwmt={matchtype}&aid={creative} As far as I can tell, Google allows you to pass the keyword, creative (ad id), network and a mobile flag. Microsoft allows you to pass the match type and and ad id, and I want to be able to do the same in my Adwords. Is this is possible, or is the only way to create different ad groups for each match type and pass a custom query? Cheers, Jez
Paid Search Marketing | | jez0000 -
Wordwatch Software: PPC Adwords campaign managers heard of, tried, or actively using this?
I've been trialing WordWatch for about a month. I'll admit I've been skeptical from the start. I don't quite understand the results they're delivering or how it works. So I did a search for "Wordwatch review" hoping someone out there could shed some light or help me decide whether this software was worth keeping. But all I can find are two suspicious and badly written posts, immediately raising red flags. (Penuguin should have eliminated crap sites using the Flesch-Kincaid reading level, but I digress.) **Wordwatch premise: **They take over keyword bidding to maximize budgets and clicks. They monitor the Adwords campaign to find an "optimal" bid price. Two questions about this premise: How is it different than using the Google settings for optimize for clicks or conversions? Since Google Adwords is based on a Vickery auction, wouldn't lowering my bid only lower my position? Bearing everyone has the same QS, then lowering my bids to the range between 2 positions does not increase my actual cost. I have Wordwatch enabled for a few of my campaigns. Their interface leaves a lot to be desired. They don't report the activity or the changes they make to the campaigns from the dashboard. I had to go into my Adwords Change History to track what they were doing. And lo and behold they're also adding long tail keywords to my ad groups. Bottom line I didn't notice any huge impact, and I don't see how it's better than Google's own version of campaign settings. I don't know that they're really legit. But their marketing was so convincing, and they raised $1.4M that I need other opinions. Any one with some pro/cons, or yay/nays?
Paid Search Marketing | | flowsimple0