Sitemaps, 404s and URL structure
-
Hi All!
I recently acquired a client and noticed in Search Console over 1300 404s, all starting around late October this year.
What's strange is that I can access the pages that are 404ing by cutting and pasting the URLs and via inbound links from other sites.
I suspect the issue might have something to do with Sitemaps. The site has 5 Sitemaps, generated by the Yoast plugin. 2 Sitemaps seem to be working (pages being indexed), 3 Sitemaps seem to be not working (pages have warnings, errors and nothing shows up as indexed). The pages listed in the 3 broken sitemaps seem to be the same pages giving 404 errors.
I'm wondering if auto URL structure might be the culprit here. For example, one sitemap that works is called newsletter-sitemap.xml, all the URLs listed follow the structure: http://example.com/newsletter/post-title
Whereas, one sitemap that doesn't work is called culture-event-sitemap.xml. Here the URLs underneath follow the structure http://example.com/post-title.
Could it be that these URLs are not being crawled / found because they don't follow the structure http://example.com/culture-event/post-title? If not, any other ideas?
Thank you for reading this long post and helping out a relatively new SEO!
-
Hi Daniel! Thanks for your question.
It's kind of hard to know what's going on without seeing your site. Feel free to PM it to me.
There's definitely a chance that this is the case, but if it's happening with Yoast it is likely a configuration issue on your site not with Yoast's technology. You may need to adjust your tag permalinks within your WordPress admin so that the URLs are correct in your sitemaps.
John
-
I'll make my question shorter and hopefully more clear...
If my Permalink structure in Wordpress is set up for a given custom post type, lets call it "culture", as: example.com/postname,
Yet with Yoast, a sitemap is automatically generated for posts tagged with "culture" that looks like example.com/culture/postname
Could that explain why posts being tagged as "culture" are showing up as 404s in Search Console?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Same URL for languages sub-directories
Hi All, I have a main domain and 9 different subdirectories for languages, example: www.example.com/page.html www.example.com/uk/page-uk.html www.example.com/es/page-es.html we are implementing hreflang tags for the languages, but we are thinking to get rid of the dashes on the languages URL: -uk or -es, so it will be: www.example.com/page.html www.example.com/uk/page.html www.example.com/es/page.hrml would this be a problem? to have same page names even if they are in different subdirectories? would we need to add canonical tags, at lease for the main domain URLs? www.kornferry.com/page.html Thank you, Rachel
Technical SEO | | RaquelSaiz0 -
Spammy 404s: Should I Worry?
One of my sites is getting a ton of spammy 404s with porno-like URLs. All of these 404s are linked from other sites that I assume also got hacked, and when I click on them, they are also 404s. So I'm assuming some spam site is tricking the Googlebot into thinking these URLs exist. But is this going to affect my site & SEO directly? Is it worth disavowing all of the sites linking to me? Is Google even considering these real links? Did these pages ever actually exist anywhere? Don't have a hacker-brain whatsoever so I need some enlightening. I've been told I shouldn't worry but it seems like something I should worry about...Any help is greatly appreciated 🙂 (I've updated to the newest Wordpress and Sucuri).
Technical SEO | | FSCInteractiveLLC0 -
Which url should i use? Thanks!
I have a question regarding how to use my url, we are a Swedish-based website which have the url, http://interimslösning.se/ (that contains the Swedish letter “ö”) so the url can also be written as http://xn--interimslsning-3pb.se/. Which of the following url should I use for my backlinks, http://interimslösning.se/ or http://xn--interimslsning-3pb.se/ ? What is the difference between them regarding SEO? And is it good or bad to use letter like "ö" or other characters like that in your url? I was thinking that maybe it is good to use the letter "ö" for local search optimization in sweden, but i don't know.. Thanks in advance! Greetings,
Technical SEO | | Kiwibananlime
Paul Linderoth0 -
Yet Another, Yet Important URL structure query.
Massive changes to our stock media site and structure here. While we have an extensive category system previously our category pages have only been our search pages with ID numbers for sorting categories. Now we have individual category pages. We have about 600 categories with about 4 max tiers. We have about 1,000,000 total products and issues with products appearing to be duplicate. Our current URL structure for producta looks like this: http://example.com/main-category/12345/product-name.htm Here is how I was planning on doing the new structure: Cat tier 1: http://example.com/category-one/ Cat tier 2: http://example.com/category-one/category-two/ Cat tier 3: http://example.com/category-one-category-two/category-three Cat tier 4: http://example.com/category-one-category-two-category-three/category-four/ Product: http://example.com/category-one-category-two-category-three/product-name-12345.htm Thoughts? Thanks! Craig
Technical SEO | | TheCraig0 -
I want to resubmit sitemap
I am doing major changes in my website some of my old url pages i don't want them to be indexed or submitted in site map some of other old pages i want to keep them and there is new pages any one can give me hints what should i do also I have thousands of pages on my website and I don't want to submit all my pages i want to submit best pages to google in sitemap that why i want to resubmit new site maps
Technical SEO | | Jamalon0 -
Is there any problem with my information structure?
Hey guyz I have a client who got a very interesting structure that I've ever seen. He has got a navigation down link, and with that he links every page on his site , with his every each page.
Technical SEO | | atakala
I mean each page link each page with dropdown navigational menu. ( Menu can be crawled .)
And the other interesting thing is in the image . http://prntscr.com/3q7zp6 He has a level 1 page that has a huge content in it.
But he links every topic of the content with another link which is anchor text link I mean this (http://site.com/level2page.html#part1).
How Google treats this ?
Is there anything wrong with it ? I mean amount of it .
Thank you! 3q7zp60 -
Should we block URL param in Webmaster tools after URL migration?
Hi, We have just released a new version of our website that now has a human readable nice URL's. Our old ugly URL's are still accessible and cannot be blocked/redirected. These old URL's use a URL param that has an xpath like expression language to define the location in our catalog. We have about 2 million pages indexed with this old URL param in it while we have approximately 70k nice URL's after the migration. This high number of old URL's is due to facetting that was done using this URL param. I wonder if we should now completely block this URL param from Google Webmaster tools so that these ugly URL's will be removed from the Google index. Or will this harm our position in Google? Thanks, Chris
Technical SEO | | eCommerceSEO0 -
Bing Sitemaps and Pending Status
Anyone know whats happening with the Bing Sitemap submissions and the Pending status? Mine has been pending for over a week now.
Technical SEO | | findachristianjob0
