Site less than 20 pages shows 1,400+ pages when crawled
-
Hello! I’m new to SEO, and have been soaking up as much as I can. I really love it, and feel like it could be a great fit for me – I love the challenge of figuring out the SEO puzzle, plus I have a copywriting/PR background, so I feel like that would be perfect for helping businesses get a great jump on their online competition.
In fact, I was so excited about my newfound love of SEO that I offered to help a friend who owns a small business on his site. Once I started, though, I found myself hopelessly confused.
The problem comes when I crawl the site. It was designed in Wordpress, and is really not very big (part of my goal in working with him was to help him get some great content added!)
Even though there are only 11 pages – and 6 posts – for the entire site, when I use Screaming Frog to crawl it, it sees HUNDREDS of pages. It stops at 500, because that is the limit for their free version. In the campaign I started here at SEOmoz, and it says over 1,400 pages have been crawled…with something like 900 errors.
Not good, right?
So I've been trying to figure out the problem...when I look closer in Screaming Frog, I can see that some things are being repeated over and over. If I sort by the Title, the URLs look like they’re stuck in a loop somehow - one line will have /blog/category/postname…the next line will have /blog/category/category/postname…and the next line will have /blog/category/category/category/postname…and so on, with another /category/ added each time.
So, with that, I have two questions
- Does anyone know what the problem is, and how to fix it?
- Do professional SEO people troubleshoot this kind of stuff all of the time? Is this the best place to get answers to questions like that? And if not, where is?
Thanks so much in advance for your help! I’ve enjoyed reading all of the posts that are available here so far, it seems like a really excellent and helpful community...I'm looking forward to the day when I can actually answer the questions!!
-
Thanks, Irving! I am trying turning on/off the plugins - the person who designed the site used a WP Boxer plugin and Multiple Content Blocks plugin, and that is how the homepage is designed (feeding info from pages/posts) so I was wondering if that could be part of it...but when I turn them off/on that doesn't seem to help. So I'm trying the other plugins too (there are just a couple), and if that doesn't work, I'll try a fresh install!
I also tried changing the permalink structure to just /sample-post/ and that didn't seem to work either...but I'm going to keep working on it!
I haven't tried the Twitter approach yet - because I don't actually have a Twitter account (I'm trying to keep social media from taking over my life) - but if that's where the answers are, I guess I need to get on there!
-
Did you install plugins that might have caused the issue? I would deactivate all plugins and see if it has an effect then turn them on one at a time to see if you can isolate the issue.
If the plugins are not the issue, it might make sense to backup the DB and do a fresh install of WP which isn't hard.
-
I don't think the site moved hosts - I'm not the person who created it, but his business is relatively new, so if there was a change it would have been done with very little content on the site.
The permalink structure is custom and looks like this: /blog/%year%/%monthnum%/%day%/%postname%/
Would something else be better? Let me know! Thanks!!
-
Hey K,
If you could post a screen shot of the Settings>Permalink structure screen in the Wordpress Dashboard, or just copy and paste whatever is written in there in a reply, that might help diagnose the issue. Also, do you know if the site has moved hosts recently and was re-installed using the Wordpress export & import feature?
-
Thanks, Alan! I'll try contacting those guys!
-
This is most likely a job for a WordPress programmer because it sounds like there's a major flaw in the site architecture. Andrew Norcross @Norcross on Twitter, and Joe Hall @JoeHall would be the two people I'd reach out to for guidance if it were me.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do i beat my spammy competitors? they are ranking on page 1 of google!
my competitor is ranking on page 1 of google. he has 3.4 million backlinks and 1400 refering domains. he has aquired these backlinks from various websites, but he doesnot have links from his niche . how do i beat my competition with less backlinks because if i follo his technique, it would a takea lot of time and people to build backlinks one of my strategy is to get .edu links second strategy is to have 6000 word content and rank for really low competition keywords related to my website.( my competitors website has 1500 words content!) any other strategy you can suggest?
Intermediate & Advanced SEO | | calvinkj0 -
Google Indexed Site A's Content On Site B, Site C etc
Hi All, I have an issue where the content (pages and images) of Site A (www.ericreynolds.photography) are showing up in Google under different domains Site B (www.fastphonerepair.com), Site C (www.quarryhillvet.com), Site D (www.spacasey.com). I believe this happened because I installed an SSL cert on Site A but didn't have the default SSL domain set on the server. You were able to access Site B and any page from Site A and it would pull up properly. I have since fixed that SSL issue and am now doing a 301 redirect from Sites B, C and D to Site A for anything https since Sites B, C, D are not using an SSL cert. My question is, how can I trigger google to re-index all of the sites to remove the wrong listings in the index. I have a screen shot attached so you can see the issue clearer. I have resubmitted my site map but I'm not seeing much of a change in the index for my site. Any help on what I could do would be great. Thanks
Intermediate & Advanced SEO | | cwscontent
Eric TeVM49b.png qPtXvME.png1 -
Site Not Indexing After 2 Weeks - PA at 1
Hi Moz Community! I'm working as a digital marketing consultant for an organization that uses us for their online registration - we do not manage their web page. The issue that I am hoping you might have some ideas on is that their SERPS still aren’t making much of a recovery since they revamped their site in mid August. I ran a MOZ campaign for them and despite that they (eventually) got all their 301s in place, they submitted an updated sitemap to Google, aren’t hitting any crawl errors, and have a working robots.txt over two-thirds of their site pages don’t seem to be indexing. MOZ is giving most of them a Page Authority of 1, and when I login to their GWT, it’s showing me that only 3 pages have been indexed of the 315 URLs submitted. I know Google doesn’t make any guarantees in index update timelines, but 2+ weeks seems like a long time 😞 Their website is https://www.northshoreymca.org/. The site has a DA of 43 but most of the pages on the main nav are still at 1. They gave me permission to share in this forum because we're really trying to figure out a recovery strategy. Any thoughts or ideas as to what might be causing this? Is there anything else that you think I should check or that might be causing an issue? Is it possible that Google is just taking this long to index their page? Note: this page is built with Drupal. THANK YOU!
Intermediate & Advanced SEO | | camarin_w0 -
SEO - Use pages on main site or set up outside keyword rich domains and websites
I have a client who is wanting to target searches for competitors products. His idea was to purchase domains related to the searches he's targeting (for example, people looking for another company's app) and to build out one page websites addressing the search query and why a customer would choose his app solution over a competitor. I know he'd have to build a handful of links to each site for any chance of success but I wanted to ask the following.. Would doing this be better than just building pages addressing the searches on his main website domain? Is there an SEO risk to doing this? Potential for a penalty? Anything we need to do to structure these in a way that won't violate Google's SEO guidelines? Any other thoughts on pros and cons of each strategy? Thank you! Ricky
Intermediate & Advanced SEO | | RickyShockley0 -
Pages with rel "next"/"prev" still crawling as duplicate?
Howdy! I have a site that is crawling as "duplicate content pages" that is really just pagination. The rel next/prev is in place and done correctly but Roger Bot and Google are both showing duplicated content + duplicate page titles & meta's respectively. The only thing I can think of is we have a canonical pointing back at the URL you are on - we do not have a view all option right now and would not feel comfortable recommending it given the speed implications and size of their catalog. Any experience, recommendations here? Something to be worried about? /collections/all?page=15"/>
Intermediate & Advanced SEO | | paul-bold0 -
How would you handle 12,000 "tag" pages on Wordpress site?
We have a Wordpress site where /tag/ pages were not set to "noindex" and they are driving 25% of site's traffic (roughly 100,000 visits year to date). We can't simply "noindex" them all now, or we'll lose a massive amount of traffic. We can't possibly write unique descriptions for all of them. We can't just do nothing or a Panda update will come by and ding us for duplicate content one day (surprised it hasn't already). What would you do?
Intermediate & Advanced SEO | | M_D_Golden_Peak1 -
Crawling error or somthing else that male my page unvisible ( Simple problem, no solved yet )
Hi, my problem isn't solved and nobody was able to answer my question: why isn't my page poltronafraubrescia.zenucchi.it indexed for the keyword poltrona frau Brescia? The same page on another domain was four on the ranking reluts... And now it redirects to the new one... An you explain me how to proceed? I trust you... Help me...
Intermediate & Advanced SEO | | guidoboem0 -
Is site:domain.com + keyword a good indicator of the quality of a page?
Are the results provided by site:domain.com + keyword a good indicator of the quality of certain pages? For example, should the first result be more relevant, have a higher number of links, etc than the second result?
Intermediate & Advanced SEO | | nicole.healthline0