Why xml generator is not detecting all my urls?
-
Hi Mozzers,
After adding 3 new pages to example.com, when generating the xml sitemap, Iwasn't able to locate those 3 new url. This is the first time it is happening. I have checked the meta tags of these pages and they are fine. No meta robots setup!
Any thoughts or idea why this is happening? how to fix this?
Thanks!
-
Thank you Guys!
I resolved the issue! I think it was a caching issue!
-
Like Matt says, could it be that there is a robots.txt file that could block these URLs? Could it be that your XML sitemap generator is using cached results for your site. Can the pages be found on the page via links on other pages?
-
Are they linked to a page that the crawler can find?
What are you generating it with? Online tools like xml-sitemaps.com act like a spider - if they can't crawl from the homepage to the new page, they simply won't find it.
Internal tools, like a Wordpress plugin, should actually find those orphan pages.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site is generating long path URLs
Hi, We've seen recently in Search Console Coverage report that website is generating long path URLs that we actually don't have.
Technical SEO | | eUniverse
Here's an example: https://autocovers.co.uk/car-mats/outdoor-basic/indoor-car-covers/shop/contact-us/shipping-delivery/about-us/about-us/indoor-car-covers/ Does anybody knows what's the issue behind it? Thanks!0 -
Changing Urls
Hi All, I have a question I hope someone can help me with. I ran a scan on a website and it has a stack of urls that are far too long. I am going through and changing the urls to shorter ones. But my question is regarding redirections. Wordpress seems to be automatically redirecting the old urls to the new ones, should i be adding a more solid 301 in as well or is the wordpress redirect enough? I ask as they dont all seem to stay redirecting Thanks in advance for the help
Technical SEO | | DaleZon2 -
URL not indexed but shows in results?
We are working on a site that has a whole section that is not indexed (well a few pages are). There is also a problem where there are 2 directories that are the same content and it is the incorrect directory with the indexed URLs. The problem is if I do a search in Google to find a URL - typically location + term then I get the URL (from the wrong directory) up there in the top 5. However, do a site: for that URL and it is not indexed! What could be going on here? There is nothing in robots or the source, and GWT fetch works fine.
Technical SEO | | MickEdwards0 -
Exact URL Match For Ranking
Has anyone else run into this issue? I have a competitor that purchases domain names for popular inner pages we are trying to rank for. We are trying to build a brand, our competitors have a lower domain authority but rank higher for inner pages in the serps with VERY little content, backlinks/seo work, they host a single page and do a re-direct to their main site. Would this be a good long term strategy? EX. We sell golf clubs our brand name is golfcity (Ex only) and we carry callaway clubs, our competitor is also building a brand but they purchased callawayclubs.net and do a re-direct. They rank on page one for keywords callaway clubs. If I do try to do this does one have an advantage over another? .com. net .org. because Ive seem them all used and rank on page 1. Thank you!!!
Technical SEO | | TP_Marketing0 -
Page URL Change
We're planning on rolling out a redesign of an existing page, and at the same time, we're looking to possibly changing the URL of the page. Currently, the URL is www.blah.com/phraseword1-phraseword2-phraseword3-phraseword4 and we're ranking top 3 in Google SERP for that 4-word phrase. The keyword phrase is something we have in our Page Title, Site Copy and the URL. Now, we are planning on simplifying the URL to below.. www.blah.com/phraseword1-phraseword2 The plan is to 301 redirect the original URL to this new URL and actually work the exact phrase into the copy a few more times. My understanding is that URL doesn't get as much weight as it does in the past, but it's still important. So my question is... How important is the URL in this case where we will continue to have it in our page title and also we'll be working more copy on to the page with the appropriate keyword? Will 301 redirect from the old URL address the issue of passing SEO value for that keyword phrase? Thanks,
Technical SEO | | JoeLin
Joe0 -
Shorter URLs
Hi Is there a real value in having the keywords in the URL structure? we could use the URL: Mybrand.com/software/tablets/ipad/supertrader.html Or instead have the CMS create the shorter version mybrand.com/supertrader.html and just optimize this page for the keyword 'supertrader ipad software'
Technical SEO | | FXDD1 -
Redirect everything from a certain url
I have a new domain (www.newdomain.com) and and an old domain (www.olddomain.com). Currently both domains are pointing (via dns nameserves) at the new site. I want to 301 everything that comes from the www.oldsite.com to www.newsite.com. I've used this htaccess code RewriteEngine On RewriteCond %{HTTP_HOST} !^www.newsite.com$
Technical SEO | | EclipseLegal
RewriteRule (.*) http://www.newsite.com/$1 [R=301,L] Which works fine and redirects if someone visits www.olddomain.com but I want it to cover everything from the old domain such as www.olddomain.com/archives/article1/ etc. So if any subpages etc are visited from the old domain its redirected to the new domain. Could someone point me in the right direction? Thanks0 -
Should I not Change the URL of Ranking Pages
My site currently ranks #1 or #2 for 2 separate pages on web design & SEO for my geographic location. The URLs are currently mysite.com/services/web-design/ and mysite.com/services/seo/ I'm redesigning my site and I'm taking out the "Services" page as I'm focusing on web design and SEO and lumping everything else into my "Internet Marketing" page. Because my pages for web design and SEO rank so well, should I keep the URL structure even though I don't have a "Services" page or should I just remove services and 301 redirect so I have mysite.com/web-design/ and mysite.com/seo/. I know doing a 301 redirect could hurt me in the short term but I'm wondering if I should just bite the bullet now and change it in favor of a better URL structure. What do you think?
Technical SEO | | JaredDetroit0