Measuring the size of a competitors website?
-
I think our website is too big, far too many indexed pages. I'd like to do some research on how big our competitors' websites are (how many indexed pages). Is there a way to do this?
Cheers,
Rhys
-
Thanks, both!
-
Xenu's Link Sleuth is free so you may want to check that out (not sure just now whether there are any limits with regard to website size) but I also recommend Screaming Frog - it's money well-spent, such a feature rich tool!
-
I highly recommend buying the license for Screaming Frog, at $100/year, you won't find a more valuable SEO tool for the money. You won't find a free (and trustworthy) that will crawl a site that large.
-
Hi Alick,
I tried that but I only have the free version so we're capped at 500 URLs. Also, the site:search provided 50,000 results, but I know we don't have that many pages. Are there any other tools?
Cheers,
Rhys
-
Hi,
Do a site: search on Google itself - like "site:google.com" - to return pages of SERPs containing the pages from your competitors site which Google has indexed.
I would also suggest you use screaming frog tool you will get more accurate value here.
Hope this helps.
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Preserving Authority of closed website
Hi, I have the opportunity to acquire a domain of a company that is closing in a similar industry who currently have a decent Authority - 51DA, 5,752 Linking Domains. I cannot do anything with the new business yet but if I acquire the domain the existing website will turn off. If I crawl the existing site and 302 redirect all pages to a holding page on said domain, will that preserve authority while I start the new business and build the new website and like-for-like pages?
Technical SEO | | Ant-Scarborough0 -
Should we reinstate the old website?
In a nut shell we had a great site that performed well and grew month on month but it perhaps looked a bit dated. A decision was taken to build a new site and the job given to a PR agency for some reason. All the titles, H1 tags, page content and url structure was changed and now the site has drop 50% of organic traffic. I've been tasked with trying to rebuild rankings but so far it's not going well. A snapshot of the old website still exists and i'm very tempted to have it reinstated in the hopes that our traffic will recover. What are your thoughts?
Technical SEO | | etienneb0 -
Can we re rank our Penalyzed website in Google?
Hello This is Maqbul, from India. I have a jobs portal blog [ bharatrecruit.com]. It was getting around 50K to 100K Views a Day and made me $100 a day. But after a few months, my competitor made negative SEO with 12,000 Spammy backlinks. Suddenly my site was hit by Google and now it is getting 200 to 300 Pageviews a day. So the question is I did not disavow bad links for a long time like 3 to 4 months. Now I disavow all the bad links but the website is not ranking. Can we re-rank this site or create another website. Please reply must. None of the bloggers can answer this. Thanks, Regards Maqbul
Technical SEO | | vinaso960 -
Is Tag Manager a good option to insert text in websites?
Is Tag Manager a good option to insert text in websites? When a website doesn't have an administration panel adding text is a very big problem.
Technical SEO | | propertyshark0 -
Reverify website
Hello, I want to disavow some really dodgy links and although the website is verifed since we see a meta tag in the source code, we no longer have the login details for the email address used to verify it so can't access the tools console. So how do I upload a disavow file (the links i got were from Ahref and Moz analysis)? Should I reverify the website using a new email address and upload a new meta tag? Is there a problem with doing this? Sorry if this is obvious!
Technical SEO | | AL123al0 -
Google does not show my website anymore
Hi All, We developed a new website for the domain: www.instral.com. Before we build the webste, the domain was indexed by google and was showed as first result on a search "Instral". Without a website! Only a super simple hostingprovider webpage. Now, with the new website, all the website pages are found in google from page 7... home page is not even in the results. When i search on "instral.com" in google, it will show my website on first result including homepage. Is there something wrong with the website or DNS settings? Or mabe some other webhosting setting... am i on a blacklist or something? Bing and Yahoo are showing better results (first page). I hope someone can help me out here...
Technical SEO | | extrememedia0 -
Bad reviews coming next to the company website, how to remove those ??
My website name www.commonsite.com (duplicate name), if i search in google with keyword common site, next to that i'm getting mouthshut bad reviews. I tried various methods till now i didnt get any improvement. Finally my doubt is my site has 6 sitelinks in the search engine. What will happen if i delete that ?? Can i get those pages results next to my main website home page results. Please clarify my doubt about sitelinks.
Technical SEO | | MadhukarSV0 -
Website has been penalized?
Hey guys, We have been link building and optimizing our website since the beginning of June 2010. Around August-September 2010, our site appeared on second page for the keywords we were targeting for around a week. They then dropped off the radar - although we could still see our website as #1 when searching for our company name, domain name, etc. So we figured we had been put into the 'google sandbox' sort of thing. That was fine, we dealt with that. Then in December 2010, we appeared on the first page for our keywords and maintained first page rankings, even moving up the top 10 for just over a month. On January 13th 2011, we disappeared from Google for all of the keywords we were targeting, we don't even come up in the top pages for company name search. Although we do come up when searching for our domain name in Google and we are being cached regularly. Before we dropped off the rankings in January, we did make some semi-major changes to our site, changing meta description, changing content around, adding a disclaimer to our pages with click tracking parameters (this is when SEOmoz prompted us that our disclaimer pages were duplicate content) so we added the disclaimer URL to our robots.txt so Google couldn't access it, we made the disclaimer an onclick link instead of href, we added nofollow to the link and also told Google to ignore these parameters in Google Webmaster Central. We have fixed the duplicate content side of things now, we have continued to link build and we have been adding content regularly. Do you think the duplicate content (for over 13,000 pages) could have triggered a loss in rankings? Or do you think it's something else? We index pages meta description and some subpages page titles and descriptions. We also fixed up HTML errors signaled in Google Webmaster Central and SEOmoz. The only other reason I think we could have been penalized, is due to having a link exchange script on our site, where people could add our link to their site and add theirs to ours, but we applied the nofollow attribute to those outbound links. Any information that will help me get our rankings back would be greatly appreciated!
Technical SEO | | bigtimeseo0