URL not indexed but shows in results?
-
We are working on a site that has a whole section that is not indexed (well a few pages are). There is also a problem where there are 2 directories that are the same content and it is the incorrect directory with the indexed URLs.
The problem is if I do a search in Google to find a URL - typically location + term then I get the URL (from the wrong directory) up there in the top 5. However, do a site: for that URL and it is not indexed! What could be going on here?
There is nothing in robots or the source, and GWT fetch works fine.
-
If you want to share a set of urls I'd be happy to take a look at it in case anything else jumps out.
-
I wouldn't say that the question is answered as such, more an issue identified. For me it looks like having a directory of URLs having a canonical set to another directory of duplicate URLs messes things up for Google.
I get virtually no visibly indexed single URLs out of around 500 URLs, the directory site: search returns the URLs. Some URLs were cached in the last day or 2, and plenty throw a 404 Google page when checking for a cached version. Seems flaky all round.
-
It appears as though they are though. You got what you need then? Your question is answered?
-
The canonical issue is identified. This is more of a "i've never seen that" day. Yes the directory Site: search returns all the URLs, but do a site: search for individual URLs and 95% are not showing as indexed.
-
The site command doesn't always show you every page that is indexed. You can:
- look to see if it has been cached (like you just did); or
- execute a specific site:domain.com/pagename.html or site:domain.com/section/ command to see if Google returns an indexed result; or
- look at Google Analytics to see if the page is receiving any search-engine-sourced page entries.
It sounds like your pages might, in fact, be indexed.
As to the wrong directory content getting indexed, I'm assuming you've no indexed one of them or assigned canonical tags indicating your strong preference. Both of these are only "suggestions" to Google. It can ignore you and when that happens, the situation like the one you describe happens.
The other thing to bear in mind is how long ago you noindexed or tagged your pages. It can take Google days, weeks, months and sometimes forever to catch up to your requested changes. You have to be patient and cross your fingers.
-
yes, a sample page is cached. It was cached today, however that URL using site: is not indexed. This URL was not showing as indexed yesterday either!
-
If you search for the page directly, can you see if a version of it has been cached?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you help by advising how to stop a URL from referring to another URL on my website with a 404 errorplease?
How to stop a URL from referring to another URL on my site. I'm getting a 404 error on a referred URL which is (https://webwritinglab.com/know-exactly-what-your-ideal-clients-want-in-8-easy-steps/[null id=43484])referred from URL (https://webwritinglab.com/know-exactly-what-your-ideal-clients-want-in-8-easy-steps/) The referred URL is the URL page that I want and I do not need it redirecting to the other URL as that's presenting a 404 error. I have tried saving the permalink in WordPress and recreated the .htaccess file and the problem is still there. Can you advise how to fix this please? Is it a case of removing the redirect? Is this advisable and how do I do that please? Thanks
Technical SEO | | Nichole.wynter20200 -
SERP result (URL) doesn't change after a 301
A couple of months ago there was a result in Google for our branded search term which wasn't the 'official' URL, actually the result shown in the SERP was www.mycompany-ip.nl. We've applied a 301 redirect of this URL to the 'official' URL which is a subdomain: department.mycompany.nl. From Google the redirect is obviously working, but up until now, I don't see Google replacing the incorrect URL by the correct URL. I am wondering what to do to make the result correct. André
Technical SEO | | ConclusionDigital0 -
Homepage not indexed
Hi, I have a problem with my website. From my PC, when I search for site:nobelcom.com the homepage of the website doesn't appear, but on other PCs (different IPs) it is ok.
Technical SEO | | Silviu
Also any keywords that usually responded with homepage, now responds with other page. Does anyone know way this is happening. It happen before the Penguin update, and after a fetch like google and send to index, I had the homepage back on serps0 -
Omitted results
Hello We are facing a loss in ranking and organic traffic from 3 months on our ecommerce website. Mostly we have lost our ranking on our product pages. These pages are gone in the "omitted results" of google. It all started 3 months ago, when we had to face a duplicate content issue due to a technical priblems with our servers: 2 other domains that we own have been pushed online on google, while they shouldn't have. They have created millions of links to our main domain in a few days, and duplicate version / redirection to our main website. We have fixed this a long time ago now. But in GWT we still see that these domains are bringing links to our main ecommerce. It has dowgraded from 36 millions links to 3 millions.... Even if today there is no link ! We have done a lot of optimizations on site like adding specific content to our most important page, rebuilding the navigation, adding microdatas, adding canonical urls on products pages that we found were very similar (we sell very technical products, and we have products that are very similar. Now we have choosen 1 product to put in canonical each time it was necessary) Bt still our products pages don't rank in google. They stay in the "omitted results". Before they were ranking very well on 1st page of google's results. And we have noticed that some adswe put on ads listing websites are now well ranked in the google's results!... Like if the ads were having more authority on the subject than our own webpages... We started to delete some of these ads. But it's not always possible. And 2-3 of them are still online. Any advice to get our most important webpages at the top on the google's results back? Regards
Technical SEO | | Poptafic0 -
Removing a staging area/dev area thats been indexed via GWT (since wasnt hidden) from the index
Hi, If you set up a brand new GWT account for a subdomain, where the dev area is located (separate from the main GWT account for the main live site) and remove all pages via the remove tool (by leaving the page field blank) will this definately not risk hurting/removing the main site (since the new subdomain specific gwt account doesn't apply to the main site in any way) ?? I have a new client who's dev area has been indexed, dev team has now prevented crawling of this subdomain but the 'the stable door was shut after the horse had already bolted' and the subdomains pages are on G's index so we need to remove the entire subdomain development area asap. So we are going to do this via the remove tool in a subdomain specific new gwt account, but I just want to triple check this wont accidentally get main site removed too ?? Cheers Dan
Technical SEO | | Dan-Lawrence0 -
How do I use only one URL
my site can be reach by both www.site.com and site.com. How do I make it only use www?
Technical SEO | | Weblion0 -
Does it really matter to maintain 301 redirect after de-indexing of old URLs?
Today, I was reading latest blog post on SEOmoz blog about. Uncrawled 301s - A Quick Fix for When Relaunches Go Too Well This is very interesting study about 301 & How it useful to maintain traffic. I'm working on eCommerce website and I have done similar stuff on my website. I have big confusion to manage 301 redirect. My website generates new URLs due to following actions. Re-write dynamic URLs. Re-launch entire website on different eCommerce platform. [osCommerce to Magento Commerce] Re-name category. Trasfer one product from one category to another category. I'm managing my 301 redirect with old practice. Excel sheet data from Google webmaster tools and set specific new URLs for redirect. Hoooo... Now, I have 8.5K redirect in htaccess... And, I'm thinking it's too much. Can we remove old 301 redirect from htaccess or not? This is big question for me. Because, all pages are not hyperlink on external website. Google have just de-indexed old URLs and indexed new URLs. So, Is it require to maintain 301 redirect after Google process?
Technical SEO | | CommercePundit0