URL not indexed but shows in results?
-
We are working on a site that has a whole section that is not indexed (well a few pages are). There is also a problem where there are 2 directories that are the same content and it is the incorrect directory with the indexed URLs.
The problem is if I do a search in Google to find a URL - typically location + term then I get the URL (from the wrong directory) up there in the top 5. However, do a site: for that URL and it is not indexed! What could be going on here?
There is nothing in robots or the source, and GWT fetch works fine.
-
If you want to share a set of urls I'd be happy to take a look at it in case anything else jumps out.
-
I wouldn't say that the question is answered as such, more an issue identified. For me it looks like having a directory of URLs having a canonical set to another directory of duplicate URLs messes things up for Google.
I get virtually no visibly indexed single URLs out of around 500 URLs, the directory site: search returns the URLs. Some URLs were cached in the last day or 2, and plenty throw a 404 Google page when checking for a cached version. Seems flaky all round.
-
It appears as though they are though. You got what you need then? Your question is answered?
-
The canonical issue is identified. This is more of a "i've never seen that" day. Yes the directory Site: search returns all the URLs, but do a site: search for individual URLs and 95% are not showing as indexed.
-
The site command doesn't always show you every page that is indexed. You can:
- look to see if it has been cached (like you just did); or
- execute a specific site:domain.com/pagename.html or site:domain.com/section/ command to see if Google returns an indexed result; or
- look at Google Analytics to see if the page is receiving any search-engine-sourced page entries.
It sounds like your pages might, in fact, be indexed.
As to the wrong directory content getting indexed, I'm assuming you've no indexed one of them or assigned canonical tags indicating your strong preference. Both of these are only "suggestions" to Google. It can ignore you and when that happens, the situation like the one you describe happens.
The other thing to bear in mind is how long ago you noindexed or tagged your pages. It can take Google days, weeks, months and sometimes forever to catch up to your requested changes. You have to be patient and cross your fingers.
-
yes, a sample page is cached. It was cached today, however that URL using site: is not indexed. This URL was not showing as indexed yesterday either!
-
If you search for the page directly, can you see if a version of it has been cached?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I deindex url parameters
Google indexed a bunch of our URL parameters. I'm worried about duplicate content. I used the URL parameter tool in webmaster to set it so future parameters don't get indexed. What can I do to remove the ones that have already been indexed? For example, Site.com/products and site.com/products?campaign=email have both been indexed as separate pages even though they are the same page. If I use a no index I'm worried about de indexing the product page. What can I do to just deindexed the URL parameter version? Thank you!
Technical SEO | | BT20090 -
What is the best practice to seperate different locations and languages in an URL? At the moment the URL is www.abc.com/ch/de. Is there a better way to structure the URL from an SEO perspective?
I am looking for a solution for using a new URL structure without using www.abc.com**/ch/de** in the URL to deliver the right languages in specific countries where more than one language are spoken commonly. I am looking forward to your ideas!
Technical SEO | | eviom0 -
Should We Index These Category Pages?
Currently we have marked category pages like http://www.yournextshoes.com/celebrities/kim-kardashian/ as follow/noindex as they essentially do not include any original content. On the other hand, for someone searching for Kim Kardashian shoes, it's a highly relevant page as we provide links to all the Kim Kardashian shoe sightings that we have covered. Should we index the category pages or leave them unindexed?
Technical SEO | | Jantaro0 -
SEOMOZ and Webmaster Tools showing Different Page Index Results
I am promoting a jewelry e-commerce website. The website has about 600 pages and the SEOMOZ page index report shows this number. However, webmaster tools shows about 100,000 indexed pages. I have no idea why this is happening and I am sure this is hurting the page rankings in Google. Any ideas? Thanks, Guy
Technical SEO | | ciznerguy1 -
Competitor in 70% of results?
I'm trying to rank in the UK for a keyword where one of the competitors takes up around 70% of the top 100 positions. How is that possible? Is this google really giving the most relevant results to searchers? I can't think that a user wants to be presented with the same site on nearly all of the top 10 pages?
Technical SEO | | SamCUK0 -
De-indexing thin content & Panda--any advantage to immediate de-indexing?
We added the nonidex, follow tag to our site about a week ago on several hundred URLs, and they are still in Google's index. I know de-indexing takes time, but I am wondering if having those URLs in the index will continue to "pandalize" the site. Would it be better to use the URL removal request? Or, should we just wait for the noindex tags to remove the URLs from the index?
Technical SEO | | nicole.healthline0 -
Why Google did not index our domain?
Hi, We launched tmart 60 days ago and submitted to google, bing, yahoo 20 days later. But google had never indexed our website still when yahoo indexed it in one week. What we have checked or tried: 1. We got 20~50 inlinks in one month and now 81 inlinks via yahoo site explorer. 2. This domain has registered for 13 years and we purchased it from sedo last year. We
Technical SEO | | zt673
did not find any problems from domain archive pages. 3. Page similar: the homepage is 50% similar to one of our competitors when we just launched.
So we adjusted the page structure and modified the content one month later and decreased the similarity to 30% (by tools from webconfs.com) 4. Google Robots: googlebot crawled our website every day after we submitted for indexing.
We opened GWT account for it and added the xml sitemap last week. GWT said nothing
was wrong except the time of page loading. Our questions: Why google did not indexed our website? What should we do? Thanks, wu0