Measuring the size of a competitors website?
-
I think our website is too big, far too many indexed pages. I'd like to do some research on how big our competitors' websites are (how many indexed pages). Is there a way to do this?
Cheers,
Rhys
-
Thanks, both!
-
Xenu's Link Sleuth is free so you may want to check that out (not sure just now whether there are any limits with regard to website size) but I also recommend Screaming Frog - it's money well-spent, such a feature rich tool!
-
I highly recommend buying the license for Screaming Frog, at $100/year, you won't find a more valuable SEO tool for the money. You won't find a free (and trustworthy) that will crawl a site that large.
-
Hi Alick,
I tried that but I only have the free version so we're capped at 500 URLs. Also, the site:search provided 50,000 results, but I know we don't have that many pages. Are there any other tools?
Cheers,
Rhys
-
Hi,
Do a site: search on Google itself - like "site:google.com" - to return pages of SERPs containing the pages from your competitors site which Google has indexed.
I would also suggest you use screaming frog tool you will get more accurate value here.
Hope this helps.
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can bad html code hurt your website from ranking ?
Hello,For example if I search for “ Bike Tours in France” I am looking for a page with a list of tours in France.Does it mean that if my html doesn’t have list * in the code but only that apparently doesn’t have any semantic meaning for a search engine my page won’t rank because of that ?Example on this page : https://bit.ly/2C6hGUn According to W3schools: "A semantic element clearly describes its meaning to both the browser and the developer. Examples of non-semantic elements: <div> and - Tells nothing about its content. Examples of semanticelements: <form>, , and- Clearly defines its content."Has anyone any experience with something similar ?Thank you, </form>
Technical SEO | | seoanalytics0 -
Website Revision
Hi all~ We are completely remaking our website: www.containmydog.com. I believe we have a good handle on the visual aspects of the redesign. What are the backend or behind the scenes (probably not using the technical term) things that need to be done so search engines know where things are. For example i know we are not going to remove some pages , change were some pages are on the site and add new pages. Is there a checklist that lists the important things to do when designing/redesigning a website? If there is not a checklist what are the things I should be asking the web person we hire?
Technical SEO | | PhotographerSteve1 -
Document.referrer Is that harmful to my website?
Someone (maybe my Competitors) open a subdomain on tumblr.com. Just like keywordxxx.tumblr.com , and use following script to redirect to my website. var s=document.referrer; if(s.indexOf("google")>0 || s.indexOf("bing")>0 || s.indexOf("yahoo")>0 || s.indexOf ("aol")>0){ self.location='mywebsiteurl'; } so , If anyone seach in google ,and click on keywordxxx.tumblr.com , this script will direct the user to my website. I have found many cases like that, is that harmful ? how can i avoid that?
Technical SEO | | sunvary0 -
Canonicalization on my website
I am kind of new to all this but I would like to understand canonicalization. I have a website which when you arrive on it is www.mysite.com but once inside and flicking back to the homepage it reverts to www.mysite.com/index.html. Should I be doing something re canonicalization? If so what? Will the link juice be diluted by having two home page versions? Thanks
Technical SEO | | FCAbroad0 -
Including videos on an eCommerce website.
Hi,
Technical SEO | | ChrisHolgate
We're about to start integrating video content on our eCommerece site in order to bolster the quantity and diversity of useful content that each page presents a potential customer. We’re talking product reviews, information relating to the differences between different products and company information. From an SEO perspective it has been suggested that Google will like this however I have a question regarding the actual method of integration. Usually I would just insert some embed code linking to the correct video on our YouTube channel however I’m really conscious that a search engine will just see the embed code, notice the page loads slightly slower and not really gain anything useful from the video file. I’m assuming that this isn’t the case but I have several questions: - Would you host on YouTube and simply insert the embed code in to the page. Would any alternative site (or even self-hosted) be preferable? - Should I be padding out either side of the embed code with a description of the video and an annotation on OUR site? - Does Google actually look at the link and ascertain the relevance of the video file to the actual page? If it does this does it gain anything useful that could help on-page SEO? These are probably all pretty basic questions for which I do apologise but I want to make sure that before spending a sizable amount of time on this that we start off with a correct integration! Many thanks0 -
Fixing a website redirect situation that resulted in drop in traffic
Hi, I'm trying to help someone fix the following situation: they had a website, www.domain.com, that was generating a steady amount of traffic for three years. They then redesigned the website a couple of months ago, and the website developer redirected the site to domain.com but did not set up analytics on domain.com. We noticed that there was a drop in traffic to www.domain.com but have no idea if domain.com is generating any traffic since analytics wasn't installed. To fix this situation, I was going to find out from the developer if there was a good reason to redirect the site. What would have prompted the developer to do this if www.domain.com had been used already for three years? Then, unless there was a good reason, I would change the redirect back to what it was before - domain.com redirecting to www.domain.com. Presumably this would allow us to regain the traffic to the site www.domain.com that was lost when the redirect was put in place. Does this sound like a reasonable course of action? Is there anything that I'm missing, or anything else that I should do in this situation? Thanks in advance! Carolina
Technical SEO | | csmm0 -
Advice on Linking to an Adult Related Website
I have a question regarding whether or not Google would penalize my main website for linking to a website that has adult content. The site I am linking to is not a porn site, rather it is a site that sells web site templates for adult related stores selling sexy toys, videos, etc. For example my site that is linking to the adult related website is here: http://www.websitetemplatedesign.com/ and the link to the site is in the footer at the bottom left which is an icon. And it links to http://www.adultsextemplates.com/ Im just looking for advice as to whether or not this could be a penalty or not. I did suffer major SERP loss in the last month and Im trying to find what I am doing that may have caused this. Any advice would be appreciated.
Technical SEO | | jmccommas0 -
Segmenting Website into XML Sitemaps
Hi all, I'm about to begin the process of chopping up a 1,000 page website into separate sitemaps. I'm going for a three tiered approach so that I can check indexation on each level for: Category, Subcategory, Product What's the easiest way to create three separate XML sitemaps for this? Thanks, Nick
Technical SEO | | NickPateman810