Noindex Pages indexed
-
I'm having problem that gogole is index my search results pages even though i have added the "noindex" metatag. Is the best thing to block the robot from crawling that file using robots.txt?
-
Thanks. Thats exactly what i needed.
-
Can you not just use href='/search.php' rel='canonical' /> in the head of all pages that serve the results?
Google will then index the original page people search from and ignore pages with query stringslike:
- /search.php?seo
- /search.php?ppc
-
Just be aware Google still has the URLs within their searches but without descriptions. They still 'index' the URL but don't actually crawl the pages.
-
Yep, that should stop them. Now just need time for Google to crawl.
You can also check WMT to see if the robots.txt is being read.
WMT > Health > Blocked URLs
-
i have added "Disallow:/search.php?" to the robots.txt file. Google seemed to be adding details to the form and then adding the results to the index. This means there are 100,000's of pages in the index that i dont want in there. Hopefully stopping the pages being crawled will help.
-
robots.txt will block it, noindex is still usually able to find via search if you look specifically for it.
User-agent: google-bot
disallow: /sampledir
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Spam pages being redirected to 404s but sill indexed
Client had a website that was hacked about a year ago. Hackers went in and added a bunch of spam landing pages for various products. This was before the site had installed an SSL certificate. After the hack, the site was purged of the hacked pages and and SLL certificate was implemented. Part of that process involved setting up a rewrite that redirects http pages to the https versions. The trouble is that the spam pages are still being indexed by Google, even months later. If I do a site: search I still see all of those spam pages come up before most of the key "real" landing pages. The thing is, the listing on the SERP are to the http versions, so they're redirecting to the https version before serving a 404. Is there any way I can fix this without removing the rewrite rule?
Technical SEO | | SearchPros1 -
Should I noindex pages on my website that are pulled from an API integration
SEO/Moz newbie here! My organisation's website (dyob.com.au), uses an API integration to pull through listings that are shown in the site search. There is a high volume of these, all of which only contain a title, image and contact information for the business. I can see these pages coming up on my Moz accounts with issues such as duplicate content (even if they are different) or no description. We don't have the capacity to fill these pages with content. Here's an example: https://www.dyob.com.au/products/nice-buns-by-yomg I am looking for a recommendation on how to treat these pages. Are they likely to be hurting the sites SEO? We do rank for some of these pages. Should they be noindex pages? TIA!
Technical SEO | | monica.arklay0 -
Switched from and HTTPS to HTTP. My home page is facing a redirect issue from the http to https. Should I no index the HTTP or find the redirect and delete it? Thank you
Switched from and HTTPS to HTTP. My home page is facing a redirect issue from the http to https. Should I no index the HTTP or find the redirect and delete it? Thank you
Technical SEO | | LandmarkRecovery20170 -
New pages need to be crawled & indexed
Hi there, When you add pages to a site, do you need to re-generate an XML site map and re-submit to Google/Bing? I see the option in Google Webmaster Tools under the "fetch as Google tool" to submit individual pages for indexing, which I am doing right now. Thanks,
Technical SEO | | SSFCU
Sarah0 -
Why google indexed pages are decreasing?
Hi, my website had around 400 pages indexed but from February, i noticed a huge decrease in indexed numbers and it is continually decreasing. can anyone help me to find out the reason. where i can get solution for that? will it effect my web page ranking ?
Technical SEO | | SierraPCB0 -
When we have 301 page is a Rel=Canonical needed or should we make 1 Noindex?
Hi, When we have a page as 301 (Permanent Redirect) is a Rel=Canonical needed or should we make 1 Noindex? Example http://www.Somename.com/blog/138760 when clicked goes to http://www.Somename.com/blogs/whenittyam Should i noindex the below pages http://www.Somename.com/blog/138760 and add Rel=Canonical Thanks
Technical SEO | | mtthompsons0 -
Why does SEOMos Pro include noindex pages?
I'm new to SEOMoz. Been digesting the crawl data and have a tonne of action items that we'll be executing on fairly soon. Love it! One thing I noticed is in some of crawl warnings include pages that expressly have the ROBOTS meta tag with the "noindex" value. Example: many of my noindex pages don't include meta descriptions. Therefore, is it safe to ignore warnings of this nature for these pages?
Technical SEO | | ChatterBlock0 -
Why is Google only indexing 3 of 8 pages?
Hi everyone, I have a small 8 page website I launched about 6 months ago. For the life of me I can not figure out why google is only indexing 3 of the 8 pages. The pages are not duplicate content in any way. I have good internal linking structure. At this time I dont have many inbound links from others, that will come in time. Am I missing something here? Can someone give me a clue? Thanks Tim Site: www.jparizonaweddingvideos.com
Technical SEO | | fasctimseo0