Competitor outranking us despite all SEO metrics in our favour

[OBG]

Hi,

We are trying to outrank a competitor of ours on Google for around 100 terms that we are both clearly targetting. Our competitor currently 'wins' in the SERPS around 70% of the time, often ranking in the top 3 positions.

The problem is, we are absolutely convinced that all onsite and offsite SEO metrics we are monitoring would suggest that our content should almost always place higher than theirs.

Here is what I know:

• All pages on both websites are rated 'A' by the Moz on-page-grader tool for the phrases we are targetting, with virtually no tech issues to be addressed on either site
• Our domain authority is 10 points higher than theirs
• Neither of us have external links to the relevant pages on our sites (although we are working on some for ours)
• Our Page Authority is considerably higher on average - often by 10 points or more
• We have considerably more linking route domains, from better sites. Also many more total external links
• The HTML title and H1 headings of our pages contain the target phrases at the beginning - our competitors are the same. Although they often ONLY include the target phrases in their title, wheras in our title they might only take up 1/3 of the total characters
• We have images optimised for the target phrases on our pages. As do they.
• We use the target phrases roughly the same number of times in our copy text, as do they

I have totally run out of ideas now for further optimising our site/pages to consistently rank better for these target phrases, although maybe these are a couple of factors that could be having an impact on the rankings:

• The structure of their website is perhaps optimised more for these target phrases - our site is much bigger, so perhaps our 100 pages are given less relevance on our site than their 100 pages? - but surely our stronger page authority would suggest otherwise?
• Perhaps Google is using page engagement statistics to determine that their site is 'betteer' than ours in terms of user appeal and engagment?

Can anyone think of something that I might have missed? Is there another major ranking factor I have perhaps neglected in my research?

I know link building strategies are a good way to approach this in the long run, but we are currently just concerned about why we are not already ranking better when clearly they are not undertaking any link building strategies of their own.

Any help or pointers here would be enormously appreciated.

Thanks

Lou

[OBG]

Thanks again to both of you for the pointers.

I just need to get stuck into this now I think. Even delving into just a coupled of the points raised by you has thrown up severel potentially very important areas for me to look to improve.

As a starting point I am going to stop obsessing over those heading SEO metrics that seem to have taken over my life.

Lou

[AlanBleiweiss]

EGOL,

Thank you for emphasizing the quality (helpfulness/human value). I only briefly mentioned it in my response, yet it really does need to be a top priority.

[AlanBleiweiss]

Lou,

"I just wanted to throw a few factors out there in order to encourage a response like yours - packed full of useful next steps for me to evalaute this further."

THAT is priceless

Pagination:

Loading all content on one page and using a "more" button to "reveal" it, is not a best practice. Individual pages need to exist for individual sub-topic based content. This is especially true since it now appears that Google, while indexing content initially hidden to users, is likely giving less value to that hidden content than content immediately seen.

Pagination is important IF it is executed properly. If you have tens of thousands of results in paginated lists, is that one paginated group, or are they split out into separate groups based on similarity of content? If it's all just one massive group, that's likely another problem to look into, since pagination is meant to be used to say "these pages all contain links to other content where the entire group comprises very similar content around one primary topic".

Internal linking should always point more to main category page destinations than individual pieces of content. It would be unnatural from a usability perspective to link more to individual pieces of content, and thus it would be bad for SEO.

5,000 or so average crawl errors - what is causing those? Are they 404s? Were they previously valid pages? If so, those typically need to not generate 404 but instead be a direct 301 to a highly relevant live page (and where internal links within the site are updated accordingly).

So many more issues to consider...

[EGOL]

Any help or pointers here would be enormously appreciated.

You are really focused on metrics.   Those metrics are good for two things (my opinion, certainly minority opinion here):  1) entertainment value; 2) diverting productive time away from the real work of running a website.

Perhaps Google is using page engagement statistics to determine that their site is 'betteer' than ours in terms of user appeal and engagment?

That is right.   Announced here ten years ago.  Take all of the time that you spend on metrics and links and start putting it into improving the website.  Have the courage to divorce yourself from these metrics for a year.  Get engaged in different battles... beating their articles, beating their images, beating their deals, beating their service.   Then look at your improved website and how many visitors are engaging it at a higher level.

You still have to pay attention to the technical and usability details explained by Alan, but if you are making genuine improvements in your website, that make it more competitive on the basis of content and benefits to visitors then your metrics will advance on their own.

[OBG]

Hi Alan,

Thaks very much for offering your thoughts on this. Really useful comments.

You're right in that I had very much over simplified my analysis of the issue. I just wanted to throw a few factors out there in order to encourage a response like yours - packed full of useful next steps for me to evalaute this further.

On of the points you raised was 'crawl efficeincy; and on further investigation I noticed that we have an awful lot of pagination on our website which users use to browse through our articles (we have 10's of thousands). However the competitor site tends to have all results on one page along with a 'show more' button. Might this be a good place to start? Looking at the cache dates of some of the results in Google, it certainly looks like the competitor pages are being crawled more often.

I also noticed from Google webmaster tools that our internal links report shows that we are giving great prominence to many of our category landing pages, rather than the article pages themselves. Does this sound like an area worh investigating?

5000 or so average crawl errors probably also isn't helping. Particularly given that we seem to only get around 6000 or so page crawled per day. Again, I;m guessing this is worth a lot of attention.

Duplicate page titles also seem to be an issue, so there is certainly lots for me to look at here.

Thanks again

Lou

[AlanBleiweiss]

You are asking some very challenging questions, and using some very limited metric comparisons to try to figure it all out. SEO is not so easy.  If it was, many sites would be in a continual state of leap-frog as they out-do each other in similar ways.

Here are just a few questions / considerations to add to your process:

1. Regardless of the number of instances of one or more keywords on a page, what is the total volume of highly relevant content on a given page? How helpful is that information in answering questions your specific target visitors are needing to have answered?  How helpful is it in being able to allow visitors to achieve a goal they came to your site to achieve?

2. How well organized is your content in regard to very similar pages being grouped together in both navigation and URL structure?  Since my reading of your question implies the competitor site is much more "tight" in its singular focus, this is a critical factor for your site to evaluate.

3. If their site is much more 'tight' in it's singular focus, how much is dilution a factor on the other pages of your site regarding topical focus and goal intent? If there is any serious dilution happening, you'd likely need even more content within that section you are comparing, to overcome that site's strength in refined singular focus.

4. What technical issues may exist on your site that you may not have considered?  Crawl efficiency, page processing speed, canonical or duplicate content confusion? There are many other questions I could list with just this one consideration.  Even if the competitor site has some worse signals among these, if any of yours are problematic enough, that alone can be a contributing factor.

5. How much higher is the quality of the inbound link footprint for your competitor in comparison to your inbound links footprint? Just having more links isn't at all a valid consideration if you don't dig deep into the quality issue?  If they have 10% of the inbound link volume, yet half or most of their inbound links are from very highly authoritative sites and you have less of those, that is another massive consideration.

Those are just starting point considerations.