Measuring the size of a competitors website?
-
I think our website is too big, far too many indexed pages. I'd like to do some research on how big our competitors' websites are (how many indexed pages). Is there a way to do this?
Cheers,
Rhys
-
Thanks, both!
-
Xenu's Link Sleuth is free so you may want to check that out (not sure just now whether there are any limits with regard to website size) but I also recommend Screaming Frog - it's money well-spent, such a feature rich tool!
-
I highly recommend buying the license for Screaming Frog, at $100/year, you won't find a more valuable SEO tool for the money. You won't find a free (and trustworthy) that will crawl a site that large.
-
Hi Alick,
I tried that but I only have the free version so we're capped at 500 URLs. Also, the site:search provided 50,000 results, but I know we don't have that many pages. Are there any other tools?
Cheers,
Rhys
-
Hi,
Do a site: search on Google itself - like "site:google.com" - to return pages of SERPs containing the pages from your competitors site which Google has indexed.
I would also suggest you use screaming frog tool you will get more accurate value here.
Hope this helps.
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Website homepage temporarily getting removed from google index
hi, website: www.snackmagic.com The home page goes out of google index for some hours and then comes back. We are not sure why our home page is getting de-indexed temporarily. This doesn't happen with other pages on our website. This has been happening intermittently in the gap of 2-3 days. Any inputs will be very useful for us to debug this issue Thanks
Technical SEO | | manikbystadium0 -
Mobile website question
Hi Mozzers, A website I manage has a mobile friendly version of their main website and a /m version as well. I was wondering if anyone had any experience in the best way of handling this? Should we just get rid of the /m version and tag the mobile friendly version? Thanks!
Technical SEO | | KarlBantleman0 -
Slow website
Hi I have just migrated from a custom written php/mysql site to a site using wordpress and woocommerce. I couldnt believe the drop in speed . I am using a few plugins for wordpress - contact forms / social sharing. and I have a few woocommerce plugins for taking payment etc. I am hosting images css's and js's on W3 Total Cache and MAXCDN hoping to speed the site up but tools at http://tools.pingdom.com/fpt sometimes show that the time between browser request and reply can be between 1 and 15 secs. I have searched all day looking for a post I read about two months ago with a tool that seems to look at server responce and redirect processing etc hoping it would help but cant find it. If anyone knows what I am talking about I would appreciate them posting a link The site is http://www.synergy-health.co.uk and an example of an inner page is http://www.synergy-health.co.uk/home/shop/alacer-emergen-c-1000-mg-vitamin-c-acai-berry-30-packets-8-4-g-each/ Any suggestions please? Perhaps I have w3total cache set wrong? Also, as the has been tanked and is in freefal iin google ranking since January would this be a good time to change the structure of Url from home/shop/product to domain.name/brand/product? Thanks in advance !
Technical SEO | | StephenCallaghan0 -
Implementing Cannonical & Alternate tags on a large website
Hi There, Our brochureware website consists of a Desktop site (www.oursite.com)and a Mobile website (m.oursite.com). I know I need to implement the alternate tags on the desktop pages and the cannonical tags on the mobile versions. However we have a huge site is there any dynamic way through javascript to have the code be generated or is it something that should be done manually page by page? Below is sample javascript a colleague completed to attempt to dynamically develop the snippet but I am unsure if bots will be able to interpret it: Alternate version: Thanks in advance Phil
Technical SEO | | Phily0 -
HUGE decrease in links since website redesign
We recently added several new pages pages to our website. These new pages were constructed on a dev site, and then pushed live. Since the new site has gone live I have seen a huge decline in links. My external followed links have dropped from 3000 to 500 and my total website links have fallen from 35,000 to 4,500. I have done some research, and I think there is a server side issue. Where multiple versions of my URL may be running. The majority of the links built were pointing to the homepage. That being said I do not have access to our in-house dev person this week, so I am trying to identify the problem myself. I have used screaming frog to crawl my site and did not see any errors which stand out. I realize I probably need to use 301 redirects to solve this problem, I just need some guidance on how to identify what I need to 301 redirect. Second question. If I move a landing page out of the global navigation but it can still be reached through other pages on the website , will this cause issues?
Technical SEO | | GladdySEO0 -
My website keeps getting hit every other month. What should I do?
Since April 2012, my website impressions has dropped about 88% according to GWT. Every other month or so, the impressions are dropping by about 30%. My total traffic (visitors, not impressions) has dropped by about 25% but now I am getting lots of junk traffic. A few of the major keywords I used to rank for are still ranking in the top 10 but only in the USA. Lots of the keywords have gone to page 2 or 3 in the US and are gone to hell in other countries. Now I know I'm mostly responsible for this mess. About 3 years ago, I hired a freelancer to write news for my blog and she did a great job for quite some time so I stopped monitoring her work for duplicated content. Unfortunately, she started to provide me with copied content after a while and did so for almost 9 months before I noticed it. I had also hired RankPoop - errr I mean RankPop - to build some backlinks and that eventually got me in trouble too. I got an unnatural links warning in GWT in July 2012. Since then, I had more than 50% of the bad links taken down. There are still lots of them but they sure account for way less than 50% of all the backlinks. I have not submitted a reconsideration request yet as I haven't compensate for all the links taken down yet. I also started adding LOTS of fresh, unique and useful content to the website. I've added near 400 articles (sometimes up to 7 or 8 articles a day) over the last 5 months. I've also set lots of the duplicated posts to "noindex" and when they're not indexed anymore, I completely removed them in order to avoid any copyright issues (some were 100% identical to the source). I keep doing this gradually to avoid 404 errors. In early March of this year, I did a complete redesign of the site. The navigation structure stayed the same and visually, the layout is quite the same although the graphical elements are much more professional and the site is much faster. As much as I would've like to avoid a complete redesign, major technical issues from the previous design (and development platform) was now forcing me to do so. Unfortunately, I updated the website design right before the last Panda update so now I don't know if the recent traffic lost is due to the new design or because of Panda... or both. Google is like a police officer who repeatedly give you tickets for the same offense, yet they won't tell you what that offense is. My website is located at http://www.thewebhostinghero.com/ Any advice is welcomed. P.S. sorry for my english, I speak french.
Technical SEO | | sbrault740 -
Any one worked on a sites.google.com/ website ?
Hi I have a client that has a sites.google.com/ website, Has anyone ever used one ? or had to do SEO on one ? Any help would be very much appreciated Thanks
Technical SEO | | tempowebdesign0